Why do you keep a personal knowledge base? What are you trying to accomplish by saving content? What insights do you want to discover? What tasks do you want to achieve?
Although high general intelligence is common among hackers, it is not the sine qua non one might expect. Another trait is probably even more important: the ability to mentally absorb, retain, and reference large amounts of ‘meaningless’ detail, trusting to later experience to give it context and meaning. A person of merely average analytical intelligence who has this trait can become an effective hacker, but a creative genius who lacks it will swiftly find himself outdistanced by people who routinely upload the contents of thick reference manuals into their brains.
I'm a researcher. Absorbing data in a "blue sky" kind of way ("no idea if i'll need this, this doesn't serve any explicit purpose, but hey it seems interesting") is a survival trait.
Being able to let go of data is similarly just as important, being able to have control over an internally oriented structure that allows for this process of accepting, selecting, absorbing, checking, retaining, and rejecting information, this is important for survival period.
- Recording and crystallizing ideas: most deep thoughts and informed opinions about things often don't materialize on first try. They are the result of many iterations, pivots and revisions. Writing them down helps one regain context quickly after leaving them for a while, which allows one to make incremental progress without being susceptible to the fallibility of memory searches. Writing is also frequently the best way to teach oneself something and to push oneself to identify gaps in knowledge and thinking.
- Sleep aid: committing ideas to (virtual) paper helps unburden them from one's active memory. This has inadvertent therapeutic effects as well. I'm one of those people who has tons of ideas racing in my head all the time, so this helps me sleep at night. On a related note, Cognitive Behavioral Therapy or CBT (for managing emotions) can also be done through written therapy.
- Business ideas/hobby ideas: list of stuff to do when the right combination of opportunities arises. You can have ideas but not all ideas are ripe for execution. Having them in a cache as one is constantly monitoring the environment helps one time their deployment better.
- Latent solutions to problems: Richard Feynman had a trick where he'd be constantly thinking about 6 problems at any given time. When an attack presented itself due to sheer chance, he was able to execute on it quickly, and make himself look like a genius when in fact he had been thinking about about the problem for a long time and was merely lying in wait for the right piece of the puzzle to come along.
- Jokes/good turns of phrases: not for the purpose of plagiarism, but sometimes one needs a bon mot or an apt phrase for a presentation or piece of writing. Having a store of such phrases to synthesize from is much more reliable than drawing from memory alone. Many good writers/presenters aren't Mozart-like geniuses who can produce polished work on-demand: many maintain disciplines like this to aid in producing quality work even when their memory fails them.
- Travel destinations: sometimes we read about an interesting place and tell ourselves "we'll visit some day" but then never do because when the next long weekend comes up, we'd have forgotten all about it. Keeping a list helps one to quickly converge on a destination when a vacation opportunity arises.
I’ve realized recently that I spend hours and hours a week reading and saving stuff to Evernote. Like I’m always in research mode. I think focused research is good for the career, but unfocused is just a distraction and waste of time. I save sooo much stuff and rarely go back to read it.
My one idea for recovering the lost/wasted time reading and saving this information is curating and publishing it online as blog posts for others to learn from. I’ve spent so much time categorizing an filtering through information out there, only to have it sitting in a private file, might as well make it public for humanity’s benefit (and my own).
This is exactly my experience too. I almost never go back to my notes. Only value creating notes is actual act of creating notes, it doesn't really matter what you use for it.
This is exactly the problem that I seem to have. I'm looking for a way to turn that unfocused research into something productive, and it's a tough problem to solve.
My reason for keeping a knowledge base is to forget less of what I learn every day.
I used to use Evernote for this but it wasn't quite working because of constant context switching and lack of spaced repetition.
So I wrote open source program called Dnote [0] and have been use it every day for the last year to store my learning. To minimize distraction, it works as browser extensions, cli, and IDE plugin. And I automated the spaced repetition by writing a cron job to send me digests every Friday.
I'm really into taking notes -- both written and typed. The tactile part of taking notes helps me register things better and the notes themselves greatly help with recall. For written notes, I use Peter Pauper notebooks (https://www.peterpauper.com/) and Staedtler pens (https://www.staedtler.com/intl/en/products/products-for-colo...). I'm a sucker for good stationery. For typed notes, I use Google Keep on my phone and the Notes app on my Mac.
Lately I've been using screen recording to create video notes. I developed an app called Outclip (https://checkoutclip.com) with my buddy. Although the app is meant for a different purpose (bug reporting) I use it to screen record as I'm doing things (like configuring an AWS service for instance) in case I have to repeat the steps later.
Refreshing the context when I resume some project.
I get my mind back in the groove from sometimes years ago by just opening my orgmode section or file.
The brain finds the connections again and the thought train picks up and off I go.
As a side note, I use orgmode and I wouldn't consider any other program (that I know of) for a specific reason -
What other setup could I be GUARANTEED of being able to pickup in 5, 10 ,30 years and have it work just as it did?
A paper notebook for sure but orgmode gives me much more (and less in some areas).
orgmode and emacs lets me INVEST my time and effort and know I won't lose the data and time because some webservice thing went down/changed business models/lost interest.
You bring up a good point about longevity. I currently use OneNote which was easy to be effective immediately taking notes with rich content, but I know there's always the risk of it shutting down forcing me to deal with the process of recovering my notes onto a different platform.
The key question when reading any article/book/whatever is the following: "if anything, what is actually important enough to remember from this article?", or in the case of PKBs: "What should be in my PKB and what should be discarded?"
By asking yourself this question explicitly and then actually writing down the key points you practice your own judgment on what is worth remembering and what is not.
Now the added benefit of a PKB is that you can actually keep these key points in a central location and reference them in other contexts or share them with others. You are basically doing prework for later (research) questions you might have. However I would say the biggest benefit is the process; explicitly writing down what is important and discarding what is not.
For more on this subject I recommend following @fortelabs on Twitter. He does a lot of interesting writing on the subject.
If anyone is interested, the term for this field is "personal knowledge management."
There are a few basic principles that can help keep the system organized and useful.
Personally I use Google Keep and OneNote: every weekend I funnel the unstructured thoughts, ideas and realizations into a structured format organized in OneNote. It takes a bit of time, but I can look back on any of my notes on programming or real estate or marketing or psychology and instantly find them useful.
Please, for the love of all that is open source, someone create an open alternative to OneNote. It’s easily one of the best Microsoft tools ever, and there doesn’t seem to be a good equivalent. It also seems like the perfect pinboard-style lifestyle business.
I used to keep a tonne of stuff in Evernote, but have since moved to Standard Notes[1]. I generally have everything super organized, but allow the overall collection of knowledge to look messy. A bit like our brains, knowledge dumps can appear messy, but are actually organized.
Emacs noob here. I came to Emacs through learning http://overtone.github.io/ and see a lot of personal knowledge management tools that use Emacs (Org mode). Can anyone recommend some engaging Emacs tutorials to help me find my bearings?
I journal to keep a health record. I know someone who had cancer and was able to prove with photos that their chemo wasn't working, the cancer was getting worse, and get their chemo changed sooner rather than later. A personal health database can be a powerful tool for health management. I also keep articles pertinent to my health, plus links to interesting HN comments, etc.
When trying to decide where to move next, I kept a lot of information on various places that were under consideration. This included not only US states and cities, but at least one other country as well.
I did a lot of therapy in my youth. I find that having stuff I wrote that I can refer back to is enormously helpful for my state of mind, for keeping my goals on track, for tracking progress, etc.
I see the world differently from most people. If I don't keep track of links to articles, comments and research, I get dismissed a lot as crazy, not knowing what the hell I am talking about, etc. and other people typically do not back me up, even if I am saying it in the same forum where I originally read the information.
So I try to keep track of certain kinds of info so that the next time I make a comment on that subject, I can back it up without having to put up with so much disrespectful crap off of people. They don't have to agree with me or see the world the way I do, but I don't care to wallow in their contempt either. Supporting links goes a long way towards preventing the worst of such behavior.
(There are probably other reasons. This answer not guaranteed to be comprehensive.)
As a side note, there is an online course called "Building a Second Brain" that shows how to create an effective personal knowledge base: https://www.buildingasecondbrain.com/
It's kinda pricey, and I'm taking the course right now. But I find it's pretty effective overall. It uses evernote as the default platform for your PKM.
About fifteen years ago I started to use a wiki as my personal knowledge base. Since then I've added several thousands pages. Last year I started using Evernote, primarily because it works on the phone syncs with my desktop, and since then I don't use the wiki much. The good thing about Evernote is that you can export the notes to xml, so if they ever stop, I still have access to my notes.
I started to use the wiki as a notebook for problems that were too complicated to solve in one day. Plus it was a good way to keep these notes together, searchable, from home and work. I started to use it for installation and configuration manuals for applications that I had to maintain. I did keep work related stuff in it, but only general things like how to setup a webserver.
My notes are the base of my day to day. I use simple txt files synced with my phone, can't stand the lag of opening OneNote and Evernote and the bugs that always deleted my items on Wunderlist. I curate everything weekly at least, try to keep the content easy to read in 10 minutes or so for the main file (tactics and strategies is the name of this file, but I put a lot information on it, more about below). I have a lot of these txt, the main ones being tasks, tactics and strategy, accounting and a lot others for personal projects, hobbies and other subjects.
The task file is the one always opened, there are my pressing issues and there I'll note anything that later I'll pass to the other files. The most important though is the tactics/strategy one, where I write details, thoughts of my life, business strategies and general stuff that I like to read at least every week, the projects I want to do next, hobbies I want to try, advice that I like to read, and even if I have some of this etched on my mind sometimes it's a great north on a confusing day. It's great to read about this idea I had a week ago and now completely forgot about because I was focused at the current issue. Writing is also great to organize the thoughts, I had so many breakthroughs just by reading and writing on these files. It's an extension of my memory and my process.
How do you go back to your tactics/strategy notes? For me, I never stumble across old ideas when they would actually be relevant, but systemic review of these ideas seems like a big hassle.
I have three modes when it comes to my career and skill development.
- learning what I need to know about my specific job (institutional knowledge). Even if my title is “Senior Developer”, in reality I usually have a fair amount of architectural level responsibilities and meetings with C level of people working at small companies. For that, I ask a lot of questions and take notes with Evernote or if I take notes on paper, I take a picture. Evernote is surprisingly good translating my writing to text for searching.
I keep a personal knowledge base so I can prepare for a meeting and be ready to answer questions.
- If I am introducing a new to the company technology, process or framework, I have a list of links that fellow developers or my manager (who is technical) to review.
- getting “interview ready”. About three months before I start seriously looking for a job,I freshen up on architectural subjects and make sure I can talk the talk. I keep a list of bookmarks and PDFs. Again, I am at a point in my career where no one asks me to do a whiteboard coding session but they do want to talk architecture. As an in the weeds developer, I know sound architecture but I don’t talk about it every day.
- I have a list of topic areas that I need to study to feel in the gaps to really consider myself a “full stack developer/architect”. But I’m usually only focused on one thing at the time. But if I find an interesting “getting started” walk through about another topic that’s on my radar, I bookmark under a folder “Things to Learn”. Right now, that’s getting deeper into AWS, Docker, NodeJS, and React.
I keep a record of any info that I learn (organized by topic) in Workflowy. I even keep a note of everything I do.
Basically my goal is to decrease the chance of going through the same process of learning something, multiple times.
I also think of it as a cache for Google. I sometimes search my knowledge base before searching Google.
For example: how to delete duplicate lines in emacs? I just found it in my knowledge base in under 3 seconds but would have taken more time googling (also more distracting)
You hit the nail on the head. I run a personal instance of Dokuwiki, and find that this is my L1 cache for knowledge. Vim, Git, Linux... all the stuff I use the most often is there and organized in the way I find optimal.
It's also very convenient to have a reference on various devices that you own. Specs, links to manuals, common troubleshooting steps and an inbox of future work and enhancements.
I don't know about others but I actually don't. I found that I am wired to remember search terms exactly as is (not saying photographically) or at least sequences of searches and I let Google do the rest. I have been able to go back to topics I had researched years ago and almost retrace my search and find what I found earlier after many years. Must be why I am a "breadth" kind of person.
Interesting that the question is why and not how. I do it because the same material (say calculus) can be presented in many different ways, but my brain learned the material by following the steps in logic that I used the first time.
I've found with several groups of material that reconnecting your brain's old steps in logic is way easier. The same material in a different way can feel interestingly very foreign.
At work I maintain a running txt file that acts as a journal, a time tracker to help with timesheets, and a scratchpad for real-time notes (meeting notes, action items, email drafts, code/config). I start a new txt file each year. Each week starts with a header with some goals for the week. Each day follows a similar pattern of highlighting priorities for the day.
Since it's txt file it opens quickly, is easily scannable and searchable. I will sometimes tag entries with phrases that I know I'm more likely to use later when I'm looking for something. The txt file is stored in dropbox and is typically open all day on whatever device I happen to be on.
I've found that tools that organize by things by folder or tag (Notational Velocity, Evernote) don't work for me - I lose track of where something went. Organizing by calendar/time allows me to remember "oh, the thing I'm looking for occurred before this thing".
[+] [-] hprotagonist|7 years ago|reply
Although high general intelligence is common among hackers, it is not the sine qua non one might expect. Another trait is probably even more important: the ability to mentally absorb, retain, and reference large amounts of ‘meaningless’ detail, trusting to later experience to give it context and meaning. A person of merely average analytical intelligence who has this trait can become an effective hacker, but a creative genius who lacks it will swiftly find himself outdistanced by people who routinely upload the contents of thick reference manuals into their brains.
I'm a researcher. Absorbing data in a "blue sky" kind of way ("no idea if i'll need this, this doesn't serve any explicit purpose, but hey it seems interesting") is a survival trait.
[+] [-] codeprimate|7 years ago|reply
[+] [-] s-shellfish|7 years ago|reply
[+] [-] dynamic99|7 years ago|reply
[+] [-] wenc|7 years ago|reply
- Recording and crystallizing ideas: most deep thoughts and informed opinions about things often don't materialize on first try. They are the result of many iterations, pivots and revisions. Writing them down helps one regain context quickly after leaving them for a while, which allows one to make incremental progress without being susceptible to the fallibility of memory searches. Writing is also frequently the best way to teach oneself something and to push oneself to identify gaps in knowledge and thinking.
- Sleep aid: committing ideas to (virtual) paper helps unburden them from one's active memory. This has inadvertent therapeutic effects as well. I'm one of those people who has tons of ideas racing in my head all the time, so this helps me sleep at night. On a related note, Cognitive Behavioral Therapy or CBT (for managing emotions) can also be done through written therapy.
- Business ideas/hobby ideas: list of stuff to do when the right combination of opportunities arises. You can have ideas but not all ideas are ripe for execution. Having them in a cache as one is constantly monitoring the environment helps one time their deployment better.
- Latent solutions to problems: Richard Feynman had a trick where he'd be constantly thinking about 6 problems at any given time. When an attack presented itself due to sheer chance, he was able to execute on it quickly, and make himself look like a genius when in fact he had been thinking about about the problem for a long time and was merely lying in wait for the right piece of the puzzle to come along.
- Jokes/good turns of phrases: not for the purpose of plagiarism, but sometimes one needs a bon mot or an apt phrase for a presentation or piece of writing. Having a store of such phrases to synthesize from is much more reliable than drawing from memory alone. Many good writers/presenters aren't Mozart-like geniuses who can produce polished work on-demand: many maintain disciplines like this to aid in producing quality work even when their memory fails them.
- Travel destinations: sometimes we read about an interesting place and tell ourselves "we'll visit some day" but then never do because when the next long weekend comes up, we'd have forgotten all about it. Keeping a list helps one to quickly converge on a destination when a vacation opportunity arises.
[+] [-] dynamic99|7 years ago|reply
[+] [-] josephjrobison|7 years ago|reply
My one idea for recovering the lost/wasted time reading and saving this information is curating and publishing it online as blog posts for others to learn from. I’ve spent so much time categorizing an filtering through information out there, only to have it sitting in a private file, might as well make it public for humanity’s benefit (and my own).
[+] [-] dominotw|7 years ago|reply
[+] [-] dynamic99|7 years ago|reply
[+] [-] robertAngst|7 years ago|reply
I feel like I read, remember what I need to google later, and I'm onto more.
Tbh, I pick topics and have goals in mind, even hobbyist goals are more fun than surfing the web.
[+] [-] rcdwealth|7 years ago|reply
Don't use the online services to store your personal information!
That is stupidest thing that can be.
First, sooner or later, your information may be stolen and distributed online just like apples: https://raidforums.com/Announcement-Database-Index-CLICK-ME
Second, find yourself a computer, device, where you can store your data offline, and not online.
I recommend reading the online article on SASS or Service As Software Substitute: https://www.gnu.org/philosophy/who-does-that-server-really-s...
and using free software, free as in freedom to do what you want with it to keep your personal knowledge.
Recommended software:
Cherrytree: https://www.giuspen.com/cherrytree/ It works just fine for taking notes.
GNU Emacs: http://www.gnu.org/s/emacs It has notes, reminders, calenders, you name it. it has Org mode and note taking features that blow your mind.
Vym mind mapping tool http://www.insilmaril.de/vym/
and plethora of notes and ming mapping tools available that free software is offering. Search for packages: https://www.hyperbola.info/
[+] [-] stockkid|7 years ago|reply
I used to use Evernote for this but it wasn't quite working because of constant context switching and lack of spaced repetition.
So I wrote open source program called Dnote [0] and have been use it every day for the last year to store my learning. To minimize distraction, it works as browser extensions, cli, and IDE plugin. And I automated the spaced repetition by writing a cron job to send me digests every Friday.
[0] - https://github.com/dnote-io/cli
[+] [-] carusooneliner|7 years ago|reply
Lately I've been using screen recording to create video notes. I developed an app called Outclip (https://checkoutclip.com) with my buddy. Although the app is meant for a different purpose (bug reporting) I use it to screen record as I'm doing things (like configuring an AWS service for instance) in case I have to repeat the steps later.
[+] [-] WhatIsDukkha|7 years ago|reply
I get my mind back in the groove from sometimes years ago by just opening my orgmode section or file.
The brain finds the connections again and the thought train picks up and off I go.
As a side note, I use orgmode and I wouldn't consider any other program (that I know of) for a specific reason -
What other setup could I be GUARANTEED of being able to pickup in 5, 10 ,30 years and have it work just as it did?
A paper notebook for sure but orgmode gives me much more (and less in some areas).
orgmode and emacs lets me INVEST my time and effort and know I won't lose the data and time because some webservice thing went down/changed business models/lost interest.
[+] [-] sumnole|7 years ago|reply
[+] [-] bryanph_|7 years ago|reply
By asking yourself this question explicitly and then actually writing down the key points you practice your own judgment on what is worth remembering and what is not.
Now the added benefit of a PKB is that you can actually keep these key points in a central location and reference them in other contexts or share them with others. You are basically doing prework for later (research) questions you might have. However I would say the biggest benefit is the process; explicitly writing down what is important and discarding what is not.
For more on this subject I recommend following @fortelabs on Twitter. He does a lot of interesting writing on the subject.
[+] [-] alexpetralia|7 years ago|reply
There are a few basic principles that can help keep the system organized and useful.
Personally I use Google Keep and OneNote: every weekend I funnel the unstructured thoughts, ideas and realizations into a structured format organized in OneNote. It takes a bit of time, but I can look back on any of my notes on programming or real estate or marketing or psychology and instantly find them useful.
[+] [-] shawn|7 years ago|reply
[+] [-] octosphere|7 years ago|reply
[1] https://standardnotes.org/
[+] [-] jonnydubowsky|7 years ago|reply
[+] [-] DoreenMichele|7 years ago|reply
When trying to decide where to move next, I kept a lot of information on various places that were under consideration. This included not only US states and cities, but at least one other country as well.
I did a lot of therapy in my youth. I find that having stuff I wrote that I can refer back to is enormously helpful for my state of mind, for keeping my goals on track, for tracking progress, etc.
I see the world differently from most people. If I don't keep track of links to articles, comments and research, I get dismissed a lot as crazy, not knowing what the hell I am talking about, etc. and other people typically do not back me up, even if I am saying it in the same forum where I originally read the information.
So I try to keep track of certain kinds of info so that the next time I make a comment on that subject, I can back it up without having to put up with so much disrespectful crap off of people. They don't have to agree with me or see the world the way I do, but I don't care to wallow in their contempt either. Supporting links goes a long way towards preventing the worst of such behavior.
(There are probably other reasons. This answer not guaranteed to be comprehensive.)
[+] [-] steveeq1|7 years ago|reply
It's kinda pricey, and I'm taking the course right now. But I find it's pretty effective overall. It uses evernote as the default platform for your PKM.
[+] [-] cocacola1|7 years ago|reply
[+] [-] hollander|7 years ago|reply
I started to use the wiki as a notebook for problems that were too complicated to solve in one day. Plus it was a good way to keep these notes together, searchable, from home and work. I started to use it for installation and configuration manuals for applications that I had to maintain. I did keep work related stuff in it, but only general things like how to setup a webserver.
[+] [-] pepicon|7 years ago|reply
The task file is the one always opened, there are my pressing issues and there I'll note anything that later I'll pass to the other files. The most important though is the tactics/strategy one, where I write details, thoughts of my life, business strategies and general stuff that I like to read at least every week, the projects I want to do next, hobbies I want to try, advice that I like to read, and even if I have some of this etched on my mind sometimes it's a great north on a confusing day. It's great to read about this idea I had a week ago and now completely forgot about because I was focused at the current issue. Writing is also great to organize the thoughts, I had so many breakthroughs just by reading and writing on these files. It's an extension of my memory and my process.
[+] [-] dynamic99|7 years ago|reply
[+] [-] naveen99|7 years ago|reply
[+] [-] scarface74|7 years ago|reply
- learning what I need to know about my specific job (institutional knowledge). Even if my title is “Senior Developer”, in reality I usually have a fair amount of architectural level responsibilities and meetings with C level of people working at small companies. For that, I ask a lot of questions and take notes with Evernote or if I take notes on paper, I take a picture. Evernote is surprisingly good translating my writing to text for searching.
I keep a personal knowledge base so I can prepare for a meeting and be ready to answer questions.
- If I am introducing a new to the company technology, process or framework, I have a list of links that fellow developers or my manager (who is technical) to review.
- getting “interview ready”. About three months before I start seriously looking for a job,I freshen up on architectural subjects and make sure I can talk the talk. I keep a list of bookmarks and PDFs. Again, I am at a point in my career where no one asks me to do a whiteboard coding session but they do want to talk architecture. As an in the weeds developer, I know sound architecture but I don’t talk about it every day.
- I have a list of topic areas that I need to study to feel in the gaps to really consider myself a “full stack developer/architect”. But I’m usually only focused on one thing at the time. But if I find an interesting “getting started” walk through about another topic that’s on my radar, I bookmark under a folder “Things to Learn”. Right now, that’s getting deeper into AWS, Docker, NodeJS, and React.
[+] [-] ne01|7 years ago|reply
Basically my goal is to decrease the chance of going through the same process of learning something, multiple times.
I also think of it as a cache for Google. I sometimes search my knowledge base before searching Google.
For example: how to delete duplicate lines in emacs? I just found it in my knowledge base in under 3 seconds but would have taken more time googling (also more distracting)
[+] [-] smivan|7 years ago|reply
It's also very convenient to have a reference on various devices that you own. Specs, links to manuals, common troubleshooting steps and an inbox of future work and enhancements.
[+] [-] flashgordon|7 years ago|reply
[+] [-] dynamic99|7 years ago|reply
[+] [-] wpasc|7 years ago|reply
I've found with several groups of material that reconnecting your brain's old steps in logic is way easier. The same material in a different way can feel interestingly very foreign.
[+] [-] hallman76|7 years ago|reply
Since it's txt file it opens quickly, is easily scannable and searchable. I will sometimes tag entries with phrases that I know I'm more likely to use later when I'm looking for something. The txt file is stored in dropbox and is typically open all day on whatever device I happen to be on.
I've found that tools that organize by things by folder or tag (Notational Velocity, Evernote) don't work for me - I lose track of where something went. Organizing by calendar/time allows me to remember "oh, the thing I'm looking for occurred before this thing".