[testing1235]

I also tutor students in the entry level C++ and Python courses (which are taken during your first two semesters as a CS student), and I must agree that a large cohort of my class is only able to program if they have ChatGPT/Claude open on one half of their screen. I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer.

I honestly think that doing an in person fake technical interview with a few easy Leetcode questions at the end of your education would be a good way to weed out those that have failed to even learn the basics of the trade.

[bruce511]

I'm so old I remember when calculators started appearing in general people's hands. Schools first banned them (what da ya mean you can't add a column of numbers by eye?) But gradually we switched over. We had a flirting interaction with log tables, and never did get to use a slide rule. I've no doubt old-school businesses were aghast at our ineptitude.

I'm so old we learned to program with giant C reference books. There was no internet, much less Google. We didn't have no fancy auto-complete, crumbs a text editor was considered advanced. Them youngsters coming to us couldn't program without Googling syntax, or using an IDE.

So yeah, sure, AI is changing the game. It's hard to evaluate students because the tools they are using are different to our experience. For decades we "make them code" as a measure of ability. In 3 years (their college experience) the toolset has changed.

Good students, good employees, are those who understand the problem and can adapt to a solution. AI is a tool that can be wielded well, or badly. Our approach to hiring will need to adapt as well. But good people are still out there, and good people make good workers.

To be honest I never was much in love with the leet code measure of hiring. Past a certain coding skill level I was more interested in the person than their ability to memorize an algorithm. Today that necessary skill level is lower, or at least harder to evaluate, but the problem-solving-mind is still the thing we're looking for.

So be careful of seeing the use of new tools as a weakness. The history of the world is littered with obsolete technologies. (Here's a sextant, where are we?) Rather see people who use tools for what they are, tools. Look for people who are curious, who see patterns, who get things done.

And to students I say, mastery of tools is a necessary step, but ultimately an uninteresting one. See beyond them. Be curious. Look under the hood. Ask questions like "is this code good enough to be running 30 years from now?" Because a huge amount of what you see now has foundations in code written a long time ago, and written well enough to stand for decades.

College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career.

[bluefirebrand]

> College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career.

You're gonna have to do a lot of work to convince me that people who only know how to drive an LLM are learning how to adapt to sweet fuck all

At least with a calculator, people still had to know the difference between addition and multiplication, in order to use the calculator correctly

[mrheosuper]

i don't think you can compare Calculator to LLM.

A calculator will always give you correct result as long as you give it correct input. This is not the case with LLM. No matter how good your prompt is, there always a chance the output is completely garbage.

[typewithrhythm]

One big problem from the hiring side is the time to evaluate someone once complex tools are involved.

[AI_beffr]

did you ever consider the idea that AI is not the same as a calculator? or consider the fact that there is no reason why there couldnt be another quantum leap next year? and another one after that?

[linguae]

I teach computer science at a community college in Silicon Valley. Even before generative AI became available to the general public, cheating has been an issue with CS programming assignments.

One way I try to disincentivize cheating on projects is by having in-class paper exams, including weekly quizzes, as well as in-class paper assignments, and making sure that these in-class assessments are weighted significantly (roughly 60% of the overall grade). No electronic devices are allowed for these assignments. This forces my students to be able to write code without being able look up things online or consult an AI tool.

I still assign take-home programming projects that take 1-2 weeks to complete; students submit compilable source code. Practical hands-on programming experience is still vital, and even though cheating is possible, the vast majority of my students want to learn and are honest.

Still, for in-person assessments, if I had the budget, I’d prefer to hand out laptops with no Internet connection and a spartan selection of software, just a text editor and the relevant compiler/interpreter. It would making grading in-class submissions easier. But since we don’t have this budget, in-class exams and exercises are the next best solution I could think of.

[bruce511]

This reply will likely sound disrespectful, but I post it not to be so, but rather to perhaps spark an alternate path.

As the world changes, education can be slowest to adapt. My father did his math on a slide rule. I was in high school as we transitioned to using calculators.

My personal take on your approach is that you're seeing this from the wrong side. Creating an artificial environment for testing suggests to me you're testing the wrong thing.

Of course most school, and college, classes devolve to testing memory. "Here's the stuff to learn, remember it enough to pass the exam." And I get it, this is the way it's always been, regardless of the uselessness of the information. Who can remember when Charles 1st was beheaded? Who can't Google it in an instant?

Programing on paper without online reference tools isn't a measure of anything, because in the real world those tools exist.

Indeed, the very notion that we should even be testing "ability to write code" is outdated. That the student can create code should be a given.

Rather an exam should test understanding, not facts. Here's 2 blocks of code, which is better and why? Here's some code, what are the things about it that concern you?

Instead of treating the use of AI (or Google, or online help, or that giant C reference book I had) as "cheating", perhaps teach and assess in a world where AI exists.

I truly do get it. Testing comprehension is hard. Testing understanding is hard. Testing to sift wheat from chaff is hard. But, and I'm being harsh here i know, testing memory as a proxy for intelligence or testing hand-code-output as a proxy for understanding code is borderline meaningless.

Perhaps in the age of AI the focus switches from 'writing code' to 'reading code'. From the ability to write to the ability to prompt, review, evaluate and so on.

Perhaps the skill that needs to be taught (to the degree that community college seeks to teach skills) needs to be programing with AI, not against it.

I say all this with respect for how hard your job is, and with my thanks that you do it at all. I also say it understanding that it's a huge burden on you that you didn't necessarily sign up for.

[CamperBob2]

I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer.

Don't lock down the computer unless you are hiring people to work in a SCIF. Instead, give candidates a brutally hard/weird problem and tell them to use any resources they can get their hands on, by fair means or foul. (They will do that anyway if you hire them.) Then watch how they deal with it.

Do they just give up and stalk off in a huff?

If they Google for answers, do they use sensible queries?

If they use AI, do their prompts show skill at getting ideas, avoiding blind alleys, and creating effective tests?

If they call their friends, see how effective they are at communicating the requirements and turning the answers into a solution. Might be management material.