[ubj]

One of my students recently came to me with an interesting dilemma. His sister had written (without AI tools) an essay for another class, and her teacher told her that an "AI detection tool" had classified it as having been written by AI with "100% confidence". He was going to give her a zero on the assignment.

Putting aside the ludicrous confidence score, the student's question was: how could his sister convince the teacher she had actually written the essay herself? My only suggestion was for her to ask the teacher to sit down with her and have a 30-60 minute oral discussion on the essay so she could demonstrate she in fact knew the material. It's a dilemma that an increasing number of honest students will face, unfortunately.

[somenameforme]

I wouldn't mind seeing education return to its roots of being about learning instead of credentialization. In an age where having a degree is increasingly meaningless in part due to many places simply becoming thinly veiled diploma treadmills (which are somehow nonetheless accredited), this is probably more important than ever. This is doubly so if the AI impact extremists end up being correct.

So why is the issue you described an issue? Because it's about a grade. And the reason that's relevant is because that credential will then be used to determine where she can to to university which, in turn, is a credential that will determine her breadth of options for starting her career, and so on. But why is this all done by credentials instead of simple demonstrations of skill? What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting. This is how you used to apply to colleges. Here [1], for instance, is Harvard's exam from 1869. If you pass it, you're in. Simple as that.

Obviously this creates a problem of institutions starting to 'teach the test', but with sufficiently broad testing I don't see this as a problem. If a writing class can teach somebody to write a compelling essay based on an arbitrary prompt, then that was simply a good writing class! As an aside this would also add a major selling point to all of the top universities that offer free educational courses online. Right now I think 'normal' people are mostly disinterested in those because of the lack of widely accepted credentials, which is just so backwards - people are actively seeking to maximize credentials over maximizing learning.

This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization.

[1] - https://graphics8.nytimes.com/packages/pdf/education/harvard...

[snickerbockers]

I still don't understand why standardized testing gets so much pushback. Having the students do their work in a controlled environment is the obvious solution to AI and many other problems related to academic integrity.

Its also the only way that students can actually be held to the same standards. When I was a freshman in college with a 3.4 highschool GPA, I was absolutely gobsmacked by how many kids with perfect >= 4.0 GPAs couldn't pass the simple algebra test that the university administered to all undergraduates as a prerequisite for taking any advanced mathematics course.

[eloisant]

> In an age where having a degree is increasingly meaningless

I wish I would agree with you, but I think that having a degree (or rather the right degree) is more important than ever.

Basically grades exist to decide who gets a laid back high paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck.

As one teacher told me once: we could have all of you practice chess, make a big tournament and you get to choose your university based on your chess ranking. It wouldn't be any less stupid than the current system.

[thfuran]

>What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting

Sure, but it takes < 1 second to read a GPA.

[alpinisme]

In a world where some but not all programs are “diploma treadmills,” you would expect that the reputation of the bad credentials would go down and the good credentials would go up. In some sense if the credentials were really being used (and not just as a perfunctory first pass elimination), you’d expect the most elite programs to have the highest signal to noise ratio. But the market doesn’t seem to respond to changes in credentialing capability (by hiring more from programs that start focusing on the “right” things to test). Instead it’s really just a background check.

[disgruntledphd2]

> This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization.

Just so we're clear, the coding tests are in addition to credentialisation. I'll never forget when I worked at Big Tech (from Ireland) and I would constantly hear recruiters talk about the OK school list (basically the Ivy league). Additionally, I remember having to check the University a candidate had attended before she had an interview with one of our directors.

He was fine with her, because she had gone to Oxford. Honestly, I'm surprised that I was able to get hired there given all this nonsense.

[andrepd]

That's an absolutely dreadful exam. Would you mind explaining the point you wanted to make with it?

[edwcross]

Always stunned by how much teachers can accuse without proof and invert the "innocent until proven guilty".

Honestly, students should have a course in "how the justice system works" (or at least should work). So should the teachers.

Student unions and similar entities should exist and be ready to intervene to help students in such situations.

This is nothing new, AI will just make this happen more often, revealing how stupid so many teachers are. But when someone spent thousands for a tool, which purports to be reliable, and is so quick to use, how can an average person resist it? The teacher is as lazy as the cheaters they intend to catch.

[trashtester]

Student unions tend to focus on all sorts of other issues, I wouldn't trust them to handle cases like this.

The only way to reliably prevent the use of AI tools without punishing innocent students is to monitor the students while they work.

Schools can either do that by having essays be written on premise, either by hand or by using computers managed by the school.

But students that are worried that they will be targeted can also do this themselves, by setting up their phone to film them while working.

And if they do this, and the teacher tries to punish someone who can prove they wrote the essay themselves, either the teacher or the school should hopefully learn that such tools can't be trusted.

[hsuduebc2]

It’s strange watching people put so much faith in these so called “AI detection tools”. Nobody really knows how they work yet they’re treated like flawless judges. In practice they’re black boxes that quietly decide who gets flagged for “fraud”, and because the tool said so everyone pretends it must be true. The result is a neat illusion that all the “cheaters” were caught, when in reality the system is mostly just picking people at random and giving the process a fake sense of certainty.

Bizzare and unfair

[icsa]

In the United States, we all used to take a required course called Civics.

We learned how government and justice worked.

[miki123211]

> Honestly, students should have a course in "how the justice system works"

And to add to that, there should be a justice system there. The idea of due process is laughable in most educational settings.

[huevosabio]

When I was in college, there was a cheating scandal for the final exam where somehow people got their hands on the hardest question of the exam.

The professor noticed it (presumably via seeing poor "show your work") and gave zero points on the question to everyone. And once you went to complain about your grade, she would ask you to explain the answer there in her office and work through the problem live.

I thought it was a clever and graceful way to deal with it.

[raincole]

I think this kind of approach is the root of (the US's) hustle culture. Instead of receiving a fair score, you get a zero and need to "hustle" and challenge your teacher.

The teacher effectively filtered out the shy boys/girls who are not brave enough to "hustle." Gracefully.

[lazyasciiart]

Only if she advertised that option somehow. I worked two jobs in college, I didn't take time off to go complain about my grades.

[smileysteve]

Lol, in 3rd grade algebra, a teacher called 2 of us in for cheating. She had us take the test again, I got the same exact horribly failing score (a 38%) and the cheater got a better score, so the teacher then knew who the cheater was. He just chose the wrong classmate to cheat of of.

[respondo2134]

Except the power imbalance: position, experience, social, etc. meant that the vast majority just took the zero and never complained or challenged the prof. Sounds like your typical out-of-touch academic who thought they were super clever.

[stonemetal12]

Story doesn't make sense.

> the final exam where somehow people got their hands on the hardest question of the exam.

They got the question but not the answer so they had to work it out before the test. They couldn't explain it later?

[j45]

This is a nice approach. The students who know the material, or even who manually prepare before seeing the prof achieve the objective of learning.

[vondur]

I agree. Most campuses use a product called Turnitin, which was originally designed to check for plagiarism. Now they claim it can detect AI-generated content with about 80% accuracy, but I don’t think anyone here believes that.

[tyleo]

I had Turn It In mark my work as plagiarism some years ago and I had to fight for it. It was clear the teacher wasn’t doing their job and blindly following the tool.

What happened is that I did a Q&A worksheet but in each section of my report I reiterated the question in italics before answering it.

The reiterated questions of course came up as 100% plagiarism because they were just copied from the worksheet.

[phh]

80% is catastrophic though. In a classroom of 30 all honest pupils, 6 will get a 0 mark because the software says its AI?

[vkou]

> but I don’t think anyone here believes that.

All it takes is one moron with power and a poor understanding of statistics.

[jimbob45]

Had a professor use this but it was student-led. We had to run it through ourselves and change our stuff enough to get a high enough mark to pass TurnItIn. Avoided the false allegations problems at least.

[seanw265]

If they are serious they should realize that "80% accuracy" is almost meaningless for this kind of classifier. They should publish a confusion matrix if they haven't already.

[ball_of_lint]

I have had Turnitin flag my work as plagiarism for quotes from the relevant text that were quite clearly indicated as quotes.

It's shit software for schools and teachers to cover their ass. Nothing more, and deserves no more attention.

[obscurette]

There have always been problems like this. I had a classmate who wrote poems and short stories since age 6. No teacher believed she wrote those herself. She became a poet, translator and writer and admitted herself later in life that she wouldn't have believed it herself.

[ako]

My son recently told me his teacher used him as an example for the class as someone who wrote a good piece himself. Teacher accused all the other students of using AI.

He also told me that he had in fact used AI, but asked AI multiple times to simplify the text, and he had entered the simplified version. He liked the first version best, but was aware his teacher would consider it written by AI.

Guess the teachers have already lost...

[FuriouslyAdrift]

There's an easy fix. All course work must be completed in class without tools.

[RobRivera]

Giving a teacher judge/jury powers on administrative punishment jist creates a tyranny of the classroom.

Bart Simpson, we need you.

[scruple]

Further, what's to stop anyone from pumping text into an LLM and then re-writing it's output to match their own style, etc.?

[WalterBright]

My high school history teacher gave me an F on my term paper. I asked him why, and he said it was "too good" for a high school student. The next day I dumped on his desk all the cited books, which were obscure and in my dad's extensive collection. He capitulated, but disliked me ever since.

[bad_haircut72]

Now imagine this but its a courtroom and you're facing 25 years

[perihelions]

https://news.ycombinator.com/item?id=14238786 ("Sent to Prison by a Software Program’s Secret Algorithms (nytimes.com)")

https://news.ycombinator.com/item?id=14285116 ('Justice.exe: Bias in Algorithmic sentencing (justiceexe.com)")

https://news.ycombinator.com/item?id=43649811 ("Louisiana prison board uses algorithms to determine eligility for parole (propublica.org)")

https://news.ycombinator.com/item?id=11753805 ("Machine Bias (propublica.org)")

[stocksinsmocks]

Family law judges, in my small experience, are so uninterested in the basic facts of a case that I would actually trust an LLM to do a better job. Not quite what you mean, but maybe there is a silver lining.

We are already (in the US) living in a system of soft social-credit scores administered by ad tech firms and non-profits. So “the algorithms says you’re guilty” has already been happening in less dramatic ways.

[GuinansEyebrows]

The legal system has never been so easy thanks to Cinco e-Trial!

https://www.youtube.com/watch?v=XL2RLTmqG4w

[FloorEgg]

Write it in something like Google docs that tracks changes and then share the link with the revision history.

If this is insufficient, then there are tools specifically for education contexts that track student writing process.

Detecting the whole essay being copied and pasted from an outside source is trivial. Detecting artificial typing patterns is a little more tricky, but also feasible. These methods dramatically increase the effort required to get away with having AI do the work for you, which diminishes the benefit of the shortcut and influences more students to do the work themselves. It also protects the honest students from false positives.

[fuzzythinker]

Thought it is a good idea at first, but can easily be defeated with typing out AI contents. One can add pauses/deletions/edits or true edits from joining ideas different AI outputs.

[lll-o-lll]

> My only suggestion was for her to ask the teacher to sit down with her and have a 30-60 minute oral discussion on the essay so she could demonstrate she in fact knew the material.

This sounds like, a good solution? It’s the exception case, so shouldn’t be constant (false positives), although I suppose this fails if everyone cheats and everyone wants to claim innocence.

[throwaway31131]

You hinted to it but at what point are you basically giving individual oral exams to the entire class for every assignment? There are surveys where 80% of high school students self report using AI on assignments.

I guess we could go back to giving exams soviet Russia style where you get a couple of questions that you have to answer orally in front of the whole class and that’s your grade. Not fun…

[bluenose69]

The oral discussion does not scale well in large classes. The solution is to stop using essays for evaluation, relying on (supervised) examinations instead.

Of course, there will be complaints from many students. However, as a prof for decades, I can say that some will prefer an exam-based solution. This includes the students who are working their way through university and don't have much time for busy-work, along with students who write their essays themselves and get lower grades than those who do not.

[randcraw]

It's not that hard to prove that you did the work and not an AI. Show your work. Explain to the teacher why you wrote what you did, why that particular approach to the narrative appealed to you and you chose that as the basis for your work. Show an outline on which the paper was based. Show rough drafts. Explain how you revised the work, where you found your references, and why you retained some sources in the paper and not others.

To wit, show the teacher that YOU did the work and not someone else. If the teacher is not willing to do this with every student they accuse of malfeasance, they need to find another job. They're lazy as hell and suck at teaching.

[beeflet]

I agree, it isn't hard! Watch:

Computer, show "my" work and explain to the teacher why "I" wrote what "I" did, describe why that particular approach to the narrative appealed to "me" and "I" chose that as the basis of "my" work. Produce an outline on which the paper could have been based and possible rough drafts, then explain how I could have revised the work to produce the final result.

[neom]

Doesn't google docs have fairly robust edit history? If I was a student these days I'd either record my screen of me doing my homework, or at least work in google docs and share the edit history.

[Verdex]

Yeah that was my thought. Although, I went a bit more paranoid with it.

If it looks like AI cheating software will be a problem for my children (and currently it has not been an issue), then I'm considering recording them doing all of their homework.

I suspect school admin only has so much appetite for dealing with an irate parent demanding a real time review of 10 hours of video evidence showing no AI cheating.

[HelloUsername]

This still leaves many options open for plagiarism (for example a second, seperate device)

[germinalphrase]

Yes. When I was an educator, reviewing version history was an obvious way to clarify if/how much students plagiarized.

[beeflet]

this is a really flimsy method and it would be trivial to write something that works around this.

[nephihaha]

I've had the same problem online for years, when I translate something people presume I am using Google Translate (even though in one case said language isn't on Google Translate — I checked!)... Or got the answer off Wikipedia.

One of the funniest things was being accused of plagiarising Wikipedia, when I'd actually written most of the Wikipedia article on said subject. The irony... Wikipedia doesn't just use unpaid labour, it ends up undermining the people who wrote it.

[gspencley]

> when I'd actually written most of the Wikipedia article on said subject. The irony... Wikipedia doesn't just use unpaid labour, it ends up undermining the people who wrote it.

Surely it would be relatively easy to offer to show the edit history to prove that you actually contributed to the article? And, by doing so, would flip the situation in your favour by demonstrating your expertise?

The fact that you should have to is pretty annoying but also fairly edge case. And if a teacher or institute refuses to review that evidence then I don't think the credential on the table worth the paper it's printed on anyway.

[lumost]

The new trick being used by some professors in college classes is to mandate a specific document editor with a history view. If the document has unusual copy/paste patterns or was written in unusual haste then they may flag it. That being said, they support use of ai in the class and have confidence the student is not able to one shot the assignment with ai.

[CuriouslyC]

"Please take this finished essay and write me a rough first draft version of it that looks like something someone might type in on the fly before editing"

[jancsika]

Seems like this could be practically addressed by teachers adopting the TSA's randomized screening. That is, roll some dice to figure out which student on a given assignment comes in either for the oral discussion or-- perhaps in higher grades-- to write the essay in realtime.

It should be way easier than TSA's goal because you don't need to stop cheaters. You instead just need to ensure that you seed skills into a minimal number of achievers so that the rest of the kids see what the real target of education looks like. Kids try their best not to learn, but when the need kicks in they learn way better spontaneously from their peers than any other method.

Of course, this all assumes an effective pre-K reading program in the first place.

[oblio]

> Of course, this all assumes an effective pre-K reading program in the first place.

Pre-k is preschool aka kindergarten?

Is this really needed? It's really stressful for kids under 5 or 6 to read and is there a big enough statistical difference in outcome enough to rob them of some of their early youth?

I started reading around 6 years old and I was probably ahead of the vast majority of kids within 6 months.

Kids starting around 6 years old have much better focus and also greatly enhanced mental abilities overall.

[WalterBright]

> Kids try their best not to learn

Often it is more work to cheat than just learn it.

[j45]

Easy if one of these options might be available to the writer:

- Write it in google docs, and share the edit history in the google docs, it is date and time stamped.

- Make a video of writing it in the google docs tab.

If this is available, and sufficient, I would pursue a written apology to remind the future detectors.

Edit: clarity

[RobRivera]

It creates another layer of arbitrary gate-keeping. I experience this in interviews too. If I need to think about the low latency response I can give to derive djikstras algo verbatim, I get accused of reading notes on another monitor, not someone who studied for success.

It is beginning to become an awful situation where these companies are selling tools that undermine the student. Education is suppose to be the great equalizer in society, not another toggle or tool for oppression.

[rkagerer]

Guess you have to videotape or screen-record yourself writing it. Oh what a world we've created :-S.

[inerte]

You mean you'll prompt Sora to create a video of you writing the essay :)

[rcv]

... until you get accused of generating that video with another AI.

[MisterTea]

Years ago in the 90's my brother wrote a short story for a fourth grade assignment and the teacher accused him of plagiarism because how could a 9 year old come up with such a vivid and elaborate story so he received a zero. I forget the details but my mother went to the school making a big stink and eventually had the zero changed to an A.

[snarf21]

The real problem here is (in this case) lazy teachers. These kind of tools should only be used to flag potential AI generation. If the teacher read the essay and thought it reflected standard work for this student, then all would be fine. Instead they are just running the tool first to be lazy and taking the tool as gospel.

This reminds me of when GPS routing devices first came onto the scene. Lots of people drove right into a lake or ocean because the device said keep going straight. (because of poorly classified multi-modal routing data)

[theptip]

Yeah, these AI detector tools are bad. (They also punish ESL as this tends to result in simpler language patterns that parse as LLM.)

The great thing about AI is that with a bit of imagination it can be used to amplify teachers too.

In this case, yes, you need to do a viva voce to convince the teacher (though I suspect they should be able to get fairly confident in 10-15 minutes).

But you could also have students convince an AI (probably in a proctored space?) if you need to scale this approach out.

[rdudek]

This stuff is getting more pervasive too. I'm working on my Master's degree right now and any code I submit, I make sure it has spelling mistakes and make it god awful because I don't want to get flagged by some 3rd party utility that checks if it was AI generated or copied from someone else.

[JTon]

Is this how you defeat AI detection? I.e. Generate a report, then manually introduce spelling and/or grammatical errors?

[motbus3]

That's an interesting point. It seems it makes cheaper to provide knowledge but more expensive to have individual assessments.

I think AI got me some brain rot as I concern to finish stuff on time and I can't bare to spend brain energy on that (and spend on it anyway because AI sucks)

[mettamage]

I would screencast the whole thing and then tell my professor that we can watch a bit together.

[hiAndrewQuinn]

I suspect this is going in the wrong direction. Telling a sandboxed AI to have a long conversation with a student to ensure they actually know what they're talking about, while giving minimal hints away, seems like the scalable solution. Let students tackle the material however they will, knowing that they will walk into class the next day and be automatically grilled on it, unaided. There's no reason a similar student couldn't have their essay fed into the AI and then asked questions about what they meant on it.

Once this becomes routine the class can become e.g. 10 minutes conversation on yesterday's topic, 40 minutes lecturing and live exercises again. Which is really just reinventing the "daily quiz" approach, but again the thing we are trying to optimize for is compliance.

[apwell23]

> how could his sister convince the teacher she had actually written the essay herself?

She can't because she didn't write the essay herself, obviously.

[_vqpz]

You joke but I would be willing to bet this is actually the case.

[teekert]

Yeah, this, but also as an adult; When you are a non-native speaker and you use AI to make things more concise and correct. The detector will go off. People may find some wording "AI-ish" (even though I replaced em-dashes with commas and told it to "avoid American waiter-like enthusiasm"). My reaction is: Ok. you want my original? Which is much harder to read and uses 2x the amount of words? Fine.

I mean, what is the problem? It's my report! I know all the ins and outs, I take full responsibility for it. I'm the one taking this to the board of directors who will grill me on all the details. I'm up for it. So why is this so "not done"? Why do you assume I let the AI do the "thinking"? I'm appalled by your lack of trust in me.

[singpolyma3]

I routinely see people accuse any writing they don't like the style or as being AI generated. There is no possible evidence for this being the case, pple are just dicks.

[Borealid]

Is it possible the same problems that cause you to write less-readable words also lead you to choose poor AI outputs?

If no, why not?

Personally I would rather read a human's output than their selection of machine outputs.

[bradgessler]

I wrote a paper about building web applications in 10th grade a long time ago. When class was out the teacher asked me to stay for a minute after everybody left. He asked in disbelief, “did you really write that paper?”

I could see why he didn’t, so I wasn’t offended or defensive and started to tell him the steps required to build web apps and explained it in a manner he could understand using analogies. Towards the end of our conversation he could see I both knew about the topic and was enthusiastic about it. I think he was still a bit shocked that I wrote that paper, but he could tell from the way I talked about it that it was authentic.

It will be interesting to see how these situations evolve as AI gets even better. I suspect assessment will be more manual and in-person.

[jstummbillig]

How is that a dilemma for the students? What are their supposed options?

[eloisant]

Accept to be unjustly marked as a cheater, or submit to a 30-60 minutes interrogation where the teacher think you're guilty and you need to prove that you're innocent.

It's only an obvious choice if you have total faith that your teacher will be fair, which you might doubt if the situation starts with "You're a cheater unless you prove me otherwise". In the worse case scenario you'll be grilled for one hour and still be marked as a cheater because you didn't convince the teacher.

[ball_of_lint]

You don't convince the teacher, you talk with the Dean.

[totetsu]

on a side note, I wonders if anyone submitting code to github is feeling the same way about the "duplication detection filter" type AI guardrails.

[hiddencost]

I seriously think the people selling AI detection tools to teachers should be sued into the ground by a coalition of state attorneys general, and that the tools should be banned in schools.

[4pkjai]

When I was a university student in 2009 a lecturer accused me of plagiarising a c++ win32 project I submitted.

It turned out he ran it through a plagiarism detector and multiple lines of code where identical to lines in their database.

It was very silly because there’s a lot of boiler plate code in win32 projects

[swah]

Vídeo of her typing the essay>?

[johanam]

edit history in Google docs is a good way to defend yourself from AI tool use accusations

[raincole]

The funny part is that Googe has all the edit history data. In other words, it's a piece of cake for them to train a model that mimics human editing process.

The only thing prevents them from doing so is the fact Google is too big to sell a "plagiarism assistant."

[Ancapistani]

I’m very tempted to write a tool that emulates human composition and types in assignments in a human-like way, just to force academia to deal with their issues sooner.

[andrewinardeer]

Ironic that one of the biggest AI companies is also the platform to offer a service to protect yourself from allegations of using it.

[globalnode]

its defamation. flimsy proof like that needs to be sued.

[wartywhoa23]

Welcome to the post-truth society so many were and are so eager to build.

[FooBarBizBazz]

If you, like AI, have been trained on real literature -- perhaps including em-dashes, as real books do -- then you might register as AI.

but if u talk like this boss i had, then obv ur a human, kthx

Great incentives. /s