I went to school as an EE and learned in college to keep written, dated lab notes in a notebook without removable pages (to be more credible as evidence).
I did this for years in my earlier career... there's something really nice about using a pen and paper to write down ideas and sketches. I've always felt it was the "linearization" of moving your pen through making a series of marks, one at a time, that helps clarify the thinking process.
Then I actually did get tangled up in a lawsuit, by shareholders against the execs of a startup I worked for, and had all my journals subpoena'ed. It was a harsh reminder that lab notes aren't really your own private sticky notes; if they are ever going to be used as evidence, they are better thought of as a continuous performance of Engineering Best Practices.
I basically stopped writing paper notes then.
I've started journaling my coding again, but electronically (mostly using plain text notes in files, with Johnny Decimal to keep them organized). It's just so helpful during the design and early implementation phase, or when working through a tricky bug hunt. But I usually delete them when I'm done with them.
I’m curious why. Was there something in your notes that implicated you or was it more of a general “working under a microscope in public view feels stifling” or something else?
This is interesting, I'd love to hear a bit of elaboration if you find the time; why delete your notes? Why do we care if they're used in a lawsuit against our bosses, assuming we're not doing anything illegal?
I imagine I'm just being naive, because I've never worked in a small startup.
A dev log or journal can be very helpful, here are 2 ways in particular that really help me:
- context switching - if you have a backburner or side project, it's easy to get pulled away from it for days or weeks (or more) at a time, and when you do make time for it, it could be just a few minutes here or there. The very last thing I do during each dev session is list the 2-3 things I hope to tackle in the next session. When I eventually make it back to the project, I can just jump right in on one of those items and not waste any time trying to get my bearings again. I slightly disagree with the author that the journal isn't a good place to track your todos - it's a great place if you are a solo dev and/or if the work is in its early stages - you want to capture quickly important ideas of things you might do later, and sometimes just writing them down helps you not work on them right now.
- "impossible" bugs, or ones that are difficult to reproduce consistently - the journal becomes the little notebook just like the detective in a TV show. You pour into it every single clue, every bit of data. That process leads to you asking yourself certain questions or thinking of things to try to flush out the bug. This journaling is especially helpful during a crisis situation where it's easy to spin your wheels, panic, waste time, etc. - that methodical act of writing things down is calming and organizing.
I do something similar to your first approach just as I'm about to context switch (either because the day has ended or I have to work on something else).
I write a small summary of what I was doing and what were my next steps at the time, and I write this either directly in the code, in paper or sometimes in a text file. I call them ENDSNAPs (end snapshots).
I do this religiously, and would recommend two things above all else:
- Keep the format and entry method simple above all else. I use Sublime Text to edit markdown files now, and have only had trouble with various more feature-rich options like Obsidian.
- Write down what comes to you, and spend as little time as possible adhering to some format devised by you or some personality guru (like the ones discussed in this great post). You'd be surprised how easy it is to read back through less-organized notes, and if you don't experiment in the moment, you'll never find the set of rules/sections/formats/guidelines/etc. that work for you!
> Keep the format and entry method simple above all else. I use Sublime Text to edit markdown files now, and have only had trouble with various more feature-rich options like Obsidian.
Ive reached the same conclusion. I use my IDE (vscode) and markdown. Unstructured notes are easy to write, and I value that over discoverability. Its the same principle as nosql vs sql, or the same benefits of a datalake!
I almost never go back and look at my notes, which I write per task. When I do I use the VSCode search. Whats more important for me is zero friction to writing them down. Its more of a “working memory “ dump than a document to share or read again.
I actually noticed the same principle at play with arc browser. I hated thinking about “where does this tab belong” every time I wanted to open a new one or put things side by side.
I’ve been keeping a daily technical log for about 12 years now. It’s nothing fancy, managed in a series of Markdown files, one file per month. At last check when converted to a PDF it was somewhere north of 5,000 pages.
On quite a few occasions it has saved me several hours (at least) of debugging when revisiting or troubleshooting some obscure code or infrastructure change I had previously made. When not sure of the particular month/year I will just grep for all occurrences of a related term until something jogs my memory.
Recently, I've started using phpbb (with a docker-compose setup from Bitnami) for keeping personal notes. I find that it works better than pretty much everything I've used before. I like that when I reply to a thread to add an update, that post raises to the top, so it's harder to lose items.
This is exactly my use-case for a fossil repo with the built-in forum, with the default provided sorting. Fleeting notes get migrated as needed to Wiki pages (capability for which is also built-in).
Demonstration of default settings: https://fossil-scm.org/forum/forum ("most recent threads" is equivalent to "most recently updated thread")
TBH, once the feature/task/project is done...you're working on something else and all those thoughts and TODOs quickly become stale.
CTRL-A + DEL is your friend, post-work.
The biggest value of keeping notes during a project is to prevent you from getting sidetracked by distractions. Write down the thought so it's out of your head, then get back to the task at hand.
DO NOT put them in your project backlog. That's how you get a 800+ list of items that get ignored and only add a huge mental weight to your backlog management. If you're going to ignore them for 3yrs and then finally delete them, why not start by deleting (not writing) them now?
After years of solving the same electron code signing issues over and over I finally started keeping notes and now instead of wasting 6 hours troubleshooting I’m down to like 30 minutes. I guess that’s progress. Agh!
This is one of the reasons I like Fossil for source control. It has a wiki and a ticketing system built into each repo. Any branch or commit can have its own wiki page. You can customize the ticketing system and use it as a roadmap. All of that lives right within the same system as the source code. It’s great.
I've never said "I wish I hadn't put all those notes inside this repo." Fossil is good, but you could probably do the same with Github or Gitlab, assuming you're willing to use the website.
I really wish I had more information about some concrete technical problems I've solved in the past. The problem I've often run into is that writing them down is tedious. I used to record myself on video using the native camera app on iOS but found that information just gets lost in those videos. That led me to create an app that allows me to record myself on video, then transcribes what I say and synthesis it using Chat GPT. I've called it Vournal.
I've been using GitHub Issues as a coding journal for a few years now and found it to work incredibly well.
I work on most of my projects in public repositories and often end up posting hundreds of issues with thousands of accumulated comments across them all.
I also have private repos which I use just for issue threads for coding journal mode entries that I'm not ready to share.
I wouldn't call it a journal, put I keep a lot of notes for myself on tricky issues. More than once it has been really helpful figuring out an issue for myself, or I can just send the note to another person.
I keep a lot of lab notes. When I cannot think of what I need to do to move forward, I write out some lab notes of what I have done and ideas for what needs to be done.
They usually aren't complex, and and often just a single line about something.
I keep lab notes for whatever personal project I am currently engaged with, lab notes for the household (repairs, updates, documented procedures on how to start the furnace, etc), lab notes for work, lab notes for the RV, lab notes for the workshop, lab notes for the home network.
Keeping informal notes on what has been done, what needs to be done, and thoughts about the doings has helped me immensely. An example of some of my project notes https://github.com/JustinLloyd/retro-chores
I wish I was better at recording what I've done at work, uni, private. Just for my own sake.
Some screenshots, notes etc of projects I've been part of. Cool to look back and reminisce about what I've done over about 20 years of programming. Most of the early stuff is lost or forgotten.
But the few things I can find from the early days I'm quite happy to rediscover. Even if it's just a page full of construction gifs and marquee tags I made as a teenager.
I've been doing this for years, and I've developed a few "inline" methods to allow the journal to flow through multiple projects and maintain cohesion in the face of my otherwise-crippling tendency to do a million things at once and think too far ahead in the future. Perhaps some will be useful to others.
Note that these all rely on vim bindings, as I use emacs-evil. The concepts are probably not too hard to implement elsewhere, though.
- There is one 'main' journal, in my home directory, and a `devlog.md` journal in
the root of every project I'm working on. These journals are linked by a series of paths so it becomes easy to `g-f` (goto-file) to jump to the devlog in question.
- The footer of these journals consists of an immediate TODO, and the links to sub-journals previously mentioned.
- at the end of each journal is a string (,./). This allows me to jump to the place where I'd just start writing (in vim/evil emacs) with a simple `/,./`. This bookmark always sits between the 'journal' component (the top half, where the mind-dumping occurs) and the directory of child projects / TODO at the footer.
- All of these are committed to version control with a nightly cron job.
I more or less do this for every task. Link to task, todo list, useful links, design decisions, etc.
The trick for me is giving myself permission to be messy and imperfect. The todo list evolves over time and often misses things or ends up containing more than necessary. Sometimes I assert things that dont end up being that relevant. Often the design decisions are a bit handwavey.
This is all ok because 90% of the value is simply writing it down as a means of thinking. The other useful aspect is keeping track of loose ends (“make sure to use test this edge case” for example) and very occasionally a search of my notes directory unearths a useful piece of information Ive saved.
But it is not a clear and concise document that can easily be read start to finish for a complete understanding of the development process and state of my branch. It just cant be (at least for me) because it would just take too much work and its already serving the purpose of being a thought exercise, unstructured memory store, and aid for context switching.
When I made the jump from tmux+vim to Doom Emacs, the learning curve was steep, and the only thing that made it manageable was to start a Google Sheet where I recorded questions that were bothering me and their answers, if/when I discovered the answer. It was a valuable process because the act of writing down the solution helped crystallize the knowledge in my brain, plus whenever I reviewed the spreadsheet, if a question remained open too long, I knew it was time to take the question to /r/emacs or to this forum.
You can see the spreadsheet if you want! (But please don't spend any time chasing down answers for me; this is an old sheet and I don't have too many open questions about emacs right now).
[+] [-] bgribble|2 years ago|reply
I did this for years in my earlier career... there's something really nice about using a pen and paper to write down ideas and sketches. I've always felt it was the "linearization" of moving your pen through making a series of marks, one at a time, that helps clarify the thinking process.
Then I actually did get tangled up in a lawsuit, by shareholders against the execs of a startup I worked for, and had all my journals subpoena'ed. It was a harsh reminder that lab notes aren't really your own private sticky notes; if they are ever going to be used as evidence, they are better thought of as a continuous performance of Engineering Best Practices.
I basically stopped writing paper notes then.
I've started journaling my coding again, but electronically (mostly using plain text notes in files, with Johnny Decimal to keep them organized). It's just so helpful during the design and early implementation phase, or when working through a tricky bug hunt. But I usually delete them when I'm done with them.
[+] [-] schneems|2 years ago|reply
I’m curious why. Was there something in your notes that implicated you or was it more of a general “working under a microscope in public view feels stifling” or something else?
[+] [-] PTOB|2 years ago|reply
[+] [-] bbor|2 years ago|reply
I imagine I'm just being naive, because I've never worked in a small startup.
[+] [-] dbrueck|2 years ago|reply
- context switching - if you have a backburner or side project, it's easy to get pulled away from it for days or weeks (or more) at a time, and when you do make time for it, it could be just a few minutes here or there. The very last thing I do during each dev session is list the 2-3 things I hope to tackle in the next session. When I eventually make it back to the project, I can just jump right in on one of those items and not waste any time trying to get my bearings again. I slightly disagree with the author that the journal isn't a good place to track your todos - it's a great place if you are a solo dev and/or if the work is in its early stages - you want to capture quickly important ideas of things you might do later, and sometimes just writing them down helps you not work on them right now.
- "impossible" bugs, or ones that are difficult to reproduce consistently - the journal becomes the little notebook just like the detective in a TV show. You pour into it every single clue, every bit of data. That process leads to you asking yourself certain questions or thinking of things to try to flush out the bug. This journaling is especially helpful during a crisis situation where it's easy to spin your wheels, panic, waste time, etc. - that methodical act of writing things down is calming and organizing.
[+] [-] alwillis|2 years ago|reply
Doing [1] by Brett Terspstra; "A command line tool for keeping track of what you’re doing and tracking what you’ve done."
NA [2] (Next Action) also by Brett Terpstra; "A command line tool for adding and listing per-project todos."
nb [3] is "a command line and local web note‑taking, bookmarking, archiving, and knowledge base application"
nb supports multiple notebooks, Git-based version control and a bunch of other things
[1]: https://brettterpstra.com/projects/doing/
[2]: https://brettterpstra.com/projects/na/
[3]: https://xwmx.github.io/nb/#home
[+] [-] Metaluim|2 years ago|reply
I write a small summary of what I was doing and what were my next steps at the time, and I write this either directly in the code, in paper or sometimes in a text file. I call them ENDSNAPs (end snapshots).
[+] [-] bbor|2 years ago|reply
- Keep the format and entry method simple above all else. I use Sublime Text to edit markdown files now, and have only had trouble with various more feature-rich options like Obsidian.
- Write down what comes to you, and spend as little time as possible adhering to some format devised by you or some personality guru (like the ones discussed in this great post). You'd be surprised how easy it is to read back through less-organized notes, and if you don't experiment in the moment, you'll never find the set of rules/sections/formats/guidelines/etc. that work for you!
[+] [-] nonethewiser|2 years ago|reply
Ive reached the same conclusion. I use my IDE (vscode) and markdown. Unstructured notes are easy to write, and I value that over discoverability. Its the same principle as nosql vs sql, or the same benefits of a datalake!
I almost never go back and look at my notes, which I write per task. When I do I use the VSCode search. Whats more important for me is zero friction to writing them down. Its more of a “working memory “ dump than a document to share or read again.
I actually noticed the same principle at play with arc browser. I hated thinking about “where does this tab belong” every time I wanted to open a new one or put things side by side.
[+] [-] Syzygies|2 years ago|reply
I found the list years later. Last entry: "Forgot to eat. Got sick."
[+] [-] wjgilmore|2 years ago|reply
On quite a few occasions it has saved me several hours (at least) of debugging when revisiting or troubleshooting some obscure code or infrastructure change I had previously made. When not sure of the particular month/year I will just grep for all occurrences of a related term until something jogs my memory.
Highly recommended.
[+] [-] gnuvince|2 years ago|reply
[+] [-] laerios|2 years ago|reply
Demonstration of default settings: https://fossil-scm.org/forum/forum ("most recent threads" is equivalent to "most recently updated thread")
Docs: https://fossil-scm.org/home/doc/trunk/www/forum.wiki
[+] [-] tommica|2 years ago|reply
[+] [-] photoGrant|2 years ago|reply
Stolen, doing!
[+] [-] jasonpeacock|2 years ago|reply
CTRL-A + DEL is your friend, post-work.
The biggest value of keeping notes during a project is to prevent you from getting sidetracked by distractions. Write down the thought so it's out of your head, then get back to the task at hand.
DO NOT put them in your project backlog. That's how you get a 800+ list of items that get ignored and only add a huge mental weight to your backlog management. If you're going to ignore them for 3yrs and then finally delete them, why not start by deleting (not writing) them now?
[+] [-] 8note|2 years ago|reply
[+] [-] cratermoon|2 years ago|reply
[+] [-] semireg|2 years ago|reply
[+] [-] philsnow|2 years ago|reply
[+] [-] randoglando|2 years ago|reply
[+] [-] smegsicle|2 years ago|reply
[+] [-] velcrovan|2 years ago|reply
[+] [-] bachmeier|2 years ago|reply
[+] [-] tvon_g|2 years ago|reply
And if anybody wants to try it out you can download it here: https://apps.apple.com/us/app/vournal-ai-video-journal/id644...
Or if you'd like free access ask me for a TestFlight invite :)
[+] [-] mlady|2 years ago|reply
[+] [-] mlady|2 years ago|reply
[+] [-] bcrosby95|2 years ago|reply
I use it as a scratchpad as I do pretty much anything. If it gets too large, I move it into my "log" directory and start anew.
I also keep a more synthesized "notes" folder.
[+] [-] simonw|2 years ago|reply
I work on most of my projects in public repositories and often end up posting hundreds of issues with thousands of accumulated comments across them all.
I also have private repos which I use just for issue threads for coding journal mode entries that I'm not ready to share.
Occasionally I'll take one of those private issues and make it public later on, for example this one: https://github.com/simonw/public-notes/issues/1
[+] [-] jjice|2 years ago|reply
[+] [-] FigurativeVoid|2 years ago|reply
[+] [-] justinlloyd|2 years ago|reply
They usually aren't complex, and and often just a single line about something.
I keep lab notes for whatever personal project I am currently engaged with, lab notes for the household (repairs, updates, documented procedures on how to start the furnace, etc), lab notes for work, lab notes for the RV, lab notes for the workshop, lab notes for the home network.
Keeping informal notes on what has been done, what needs to be done, and thoughts about the doings has helped me immensely. An example of some of my project notes https://github.com/JustinLloyd/retro-chores
I've got lab notes going back to at least the 1970's, https://justinlloyd.li/blog/word-search-game/ (1978) and https://justinlloyd.li/blog/word-search-game-part-two/ and https://justinlloyd.li/blog/better-date-format/ (1979) are some samples, though I didn't call them lab notes back then.
[+] [-] matsemann|2 years ago|reply
Some screenshots, notes etc of projects I've been part of. Cool to look back and reminisce about what I've done over about 20 years of programming. Most of the early stuff is lost or forgotten.
But the few things I can find from the early days I'm quite happy to rediscover. Even if it's just a page full of construction gifs and marquee tags I made as a teenager.
[+] [-] pksebben|2 years ago|reply
Note that these all rely on vim bindings, as I use emacs-evil. The concepts are probably not too hard to implement elsewhere, though.
- There is one 'main' journal, in my home directory, and a `devlog.md` journal in the root of every project I'm working on. These journals are linked by a series of paths so it becomes easy to `g-f` (goto-file) to jump to the devlog in question.
- The footer of these journals consists of an immediate TODO, and the links to sub-journals previously mentioned.
- at the end of each journal is a string (,./). This allows me to jump to the place where I'd just start writing (in vim/evil emacs) with a simple `/,./`. This bookmark always sits between the 'journal' component (the top half, where the mind-dumping occurs) and the directory of child projects / TODO at the footer.
- All of these are committed to version control with a nightly cron job.
[+] [-] nonethewiser|2 years ago|reply
The trick for me is giving myself permission to be messy and imperfect. The todo list evolves over time and often misses things or ends up containing more than necessary. Sometimes I assert things that dont end up being that relevant. Often the design decisions are a bit handwavey.
This is all ok because 90% of the value is simply writing it down as a means of thinking. The other useful aspect is keeping track of loose ends (“make sure to use test this edge case” for example) and very occasionally a search of my notes directory unearths a useful piece of information Ive saved.
But it is not a clear and concise document that can easily be read start to finish for a complete understanding of the development process and state of my branch. It just cant be (at least for me) because it would just take too much work and its already serving the purpose of being a thought exercise, unstructured memory store, and aid for context switching.
[+] [-] john-radio|2 years ago|reply
You can see the spreadsheet if you want! (But please don't spend any time chasing down answers for me; this is an old sheet and I don't have too many open questions about emacs right now).
https://docs.google.com/spreadsheets/d/1BcEsENMSLmOfsjTORHLt...