Former student and TA here. There is so much great stuff in this class and its assignments! Couple of highlights:
- Data lab formally verifies all of the submissions and gives you an exact failing test case if the submission isn't correct.
- Malloc lab is run competitively. You don't need to compete to do well, but you do if you want to do great. And the competition drives students to great feats. I know people who rewrote their inner loops in assembly to get better perf. Another built splay trees to get lower fragmentation. They both had comfortable A's in the class at the time.
- The automated grading of assignments meant that the class could scale to many hundreds of students while the TAs could dedicate their time to office hours. Turnaround time for exams was sub-one-day.
- The instructors are wonderful (and well dressed!) people and they ran the class expertly.
What's the current status on accurately grading mistakes? From the exams I remember (Fall 2015), the exams autograded perfect answers well, but gave similar marks to nearly correct, but still wrong answers as complete gibberish. I felt that unless I had the answers exactly correct, it was hard to demonstrate my knowledge of the topic.
I loved this class at CMU. Recently I did shell lab again in Rust (trying to learn Rust) and used the test scripts for sanity checks. 213 helps me to this day :)
I remember when my University copied a couple of these, I decided to go the full mile on the one where you exploit buffer overflow. I got myself a remote connection via execve, and was going to mess around, but soon realized I could also see and edit others' scores which could get me in significant trouble, so I had to make one log-scrubbing attack then stop. I'm surprised they never fixed the hole by containerizing it or something.
I ran an in-class test/competition once where students submitted C code to a web server running on my laptop. My laptop compiled and ran their code - then checked it against a test suite and graded it all live.
One student asked about security - and we got into a great discussion about permissions and what havoc he could and couldn’t pull off with the restricted user account. I encouraged him to figure out how to write a fork bomb - which he got working after the test. He was nervous about it because he didn’t want to get in trouble. With some reassurances he dove into API documentation and got it working. My computer totally died and needed a reboot. It was a great little teaching opportunity that I’m glad I didn’t pass up
CS:APP is hands down THE book that every person interested in the low-level stuff in computer systems should own. But heads up, anyone who is considering to buy the book - please get the North American Edition, NOT the global edition.
I made the mistake of getting the Global edition, because of its considerably less cost, and because I couldn't afford the North American one - it was only after that I checked out the book site, where the authors mention that the global edition is chock full of errors [0].
I don't blame the authors, nor even the people who were responsible for 'the generation of a different set of practice and homework problems'. I can get printing the book in B&W, reducing paper quality, and publishing as a paperback to cut costs, but it's baffling why the publishers compromise on the actual quality of the content itself.
Amazon is full of similar 'PSAs' about not buying the global edition [1].
If I recall correctly, I believe the computer architecture course at USC (ducks) also uses these exact CS:APP labs. I loved them, it was fun realizing partway through some of the earlier labs which type of data structure they were having you walk. Decent prep for OS, if you really dig into the malloc lab.
13 years on, I continue to reference this class as the gold standard in lab design & execution. It’s the one class I continue to tell stories about to this day.
My professor for my senior operating systems class who was a CMU PhD grad, gave us these assignments (with the permission of CMU). They were some of the most important learning experiences in my undergraduate career.
Didn't go to CMU, but I did take cs61 at Harvard extension last year (https://cs61.seas.harvard.edu/site/2018/). It too had some self grading assignments (time bomb). Thoroughly enjoyed that class.
Is there a way for non-CMU students to take complete these assignments and get scored? I worked through this book a few years back but would love the chance to do actual projects.
I worked through the book and did the actual projects, if I remember well, most of the grading logic can be run locally on your machine and is included in the projects.
Former head TA (Jack) here - 15-213 still remains incredibly foundational to the work I do for my job. Working with the professors of that course (who also wrote the CS:APP textbook, which I still regret never getting signed by them!) was an amazing opportunity and something I hope I never forget.
If you want some exercises to help you learn the foundations of computer systems, I honestly cannot think of a better resource than this!
When I took this class many, many moons ago, the close equivalent of the "cache lab" was graded not on the number of cache misses but on the cache hit ratio. When I pointed the problem with this metric out to a TA he blew me off. So I turned in the original code plus a loop that read the same volatile value a few billion times. It took a long time for the cache simulator to grade, but they dutifully gave me a very high grade (something like 200%) on the lab :-)
[+] [-] abhayb|6 years ago|reply
- Data lab formally verifies all of the submissions and gives you an exact failing test case if the submission isn't correct.
- Malloc lab is run competitively. You don't need to compete to do well, but you do if you want to do great. And the competition drives students to great feats. I know people who rewrote their inner loops in assembly to get better perf. Another built splay trees to get lower fragmentation. They both had comfortable A's in the class at the time.
- The automated grading of assignments meant that the class could scale to many hundreds of students while the TAs could dedicate their time to office hours. Turnaround time for exams was sub-one-day.
- The instructors are wonderful (and well dressed!) people and they ran the class expertly.
[+] [-] albntomat0|6 years ago|reply
What's the current status on accurately grading mistakes? From the exams I remember (Fall 2015), the exams autograded perfect answers well, but gave similar marks to nearly correct, but still wrong answers as complete gibberish. I felt that unless I had the answers exactly correct, it was hard to demonstrate my knowledge of the topic.
[+] [-] swiley|6 years ago|reply
>- The automated grading of assignments meant that the class could scale to many hundreds of students
When you have things like this why are we paying so much?
[+] [-] yuji|6 years ago|reply
[+] [-] mav3rick|6 years ago|reply
[+] [-] dooglius|6 years ago|reply
[+] [-] josephg|6 years ago|reply
One student asked about security - and we got into a great discussion about permissions and what havoc he could and couldn’t pull off with the restricted user account. I encouraged him to figure out how to write a fork bomb - which he got working after the test. He was nervous about it because he didn’t want to get in trouble. With some reassurances he dove into API documentation and got it working. My computer totally died and needed a reboot. It was a great little teaching opportunity that I’m glad I didn’t pass up
[+] [-] throwaway_7718|6 years ago|reply
I made the mistake of getting the Global edition, because of its considerably less cost, and because I couldn't afford the North American one - it was only after that I checked out the book site, where the authors mention that the global edition is chock full of errors [0].
I don't blame the authors, nor even the people who were responsible for 'the generation of a different set of practice and homework problems'. I can get printing the book in B&W, reducing paper quality, and publishing as a paperback to cut costs, but it's baffling why the publishers compromise on the actual quality of the content itself.
Amazon is full of similar 'PSAs' about not buying the global edition [1].
[0] http://csapp.cs.cmu.edu/3e/errata.html
[1] https://www.amazon.com/Computer-Systems-Programmers-Perspect...
[+] [-] carlosdp|6 years ago|reply
[+] [-] ogdrhr|6 years ago|reply
[+] [-] btucker|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] agiamas|6 years ago|reply
[+] [-] mips_avatar|6 years ago|reply
[+] [-] akarki15|6 years ago|reply
[+] [-] zerr|6 years ago|reply
[+] [-] jamestimmins|6 years ago|reply
[+] [-] mathieubordere|6 years ago|reply
[+] [-] jibcage|6 years ago|reply
If you want some exercises to help you learn the foundations of computer systems, I honestly cannot think of a better resource than this!
[+] [-] voidmain|6 years ago|reply
[+] [-] westurner|6 years ago|reply
Dockerfiles might be helpful and easy to keep updated. Alpine Linux or just busybox are probably sufficient?
The instructor set could extend FROM the assignment image and run a few tests with e.g. testinfra (pytest)
You can also test code written in C with gtest.
I haven't read through all of the materials: are there suggested (automated) fuzzing tools? Does OSS-Fuzz solve?
Are there references to CWE and/or the SEI CERT C Coding Standard rules? https://wiki.sei.cmu.edu/confluence/plugins/servlet/mobile?c...
"How could we have changed our development process to catch these bugs/vulns before release?"
"If we have 100% [...] test coverage, would that mean we've prevented these vulns?"
What about 200%?
[+] [-] yjftsjthsd-h|6 years ago|reply
[+] [-] doktrin|6 years ago|reply
Also worth noting the plagiarism detector actually did its job, at least in some of the most egregious cases that I heard of.
[+] [-] proy24|6 years ago|reply
[+] [-] ec109685|6 years ago|reply
For the bomb one, are there protections for taking the binary and running it in an isolated environment?
[+] [-] MiroF|6 years ago|reply
Running it through a debugger is easy enough that this would be sorta unnecessary
[+] [-] closeparen|6 years ago|reply
[+] [-] rogela|6 years ago|reply
[+] [-] exogeny|6 years ago|reply
213, man. It's a killer.
[+] [-] Iamhisalt|6 years ago|reply
[deleted]
[+] [-] PEJOE|6 years ago|reply
[+] [-] lasagnaphil|6 years ago|reply