WingNews logo WingNews GitHub [2]
top | item 20007108

Ask HN: Do you keep a personal knowledge repository?

136 points| lethologica | 6 years ago

And if so, which software do you use? I'm fascinated by storing things that I learn outside of my own brain. Ideally I'd love to create my own mini intranet of information, basically a personal wiki, that I can go to first before having to Google things, especially for my field of study.

I've looked into various Wiki packages but none quite seem to offer what I'm looking for (Mac, no need for a webserver, cross device functionality, math markup, searchability, tags)

If you do keep a personal knowledge repository, can you please describe your workflow for using and maintaining one for all the new knowledge you acquire?

123 comments

order
[+] jjav|6 years ago|reply
How long would you like to be able to access this content?

If it is for the long haul, plain text is the only way to go (org-mode, etc).

Any proprietary commercial solution will fade away in a handful of years. At best, you might be able to extract the content before they shut down but structure and metadata may be lost.

Open source self-hosted solutions will last much longer but even in that case it might get to a point the source has been abandoned so long that it becomes difficult to keep going.

Only plain text will survive. I have plenty of notes in a more or less organized directory tree, dating back to the late 80s.

[+] tomjen3|6 years ago|reply
I am wondering what information from 1988 might still be of help for you now, but I would suggest that xml and json or similar would still last about as well as plain text, but would enable you to do more interesting things and store more useful documents (say a complete contact list).
[+] vanoora|6 years ago|reply
You can check out Anki[1]. Apart from the initial setup, you won't have to worry about taking care of the web server.

I create separate decks for different fields and then only if necessary do I create subdecks. I also tag all the cards that I create and make sure to spend a lot of time to ensure that it's made and tagged properly. So, whenever I learn something new and I believe at that point of time that it's worth remembering, I create a card for that.

[1]: http://ankiweb.net/

[+] bbischof|6 years ago|reply
I love anki, and I love this use of it, but how do you quickly make cards? Especially on mobile?
[+] james_s_tayler|6 years ago|reply
This and Workflowy are my go-to for personal knowledge repositories. Anki is such a secret weapon. I'm finding Workflowy to be another.
[+] BeetleB|6 years ago|reply
And if you use org-mode, there's org-drill (similar concepts).
[+] ashton314|6 years ago|reply
Deft in Emacs. [1] I write my notes in Markdown or Org-mode, and Deft lets me search them all really fast. For cross-file references, I use Org’s delightful file hyperlink scheme. For export, I use Pandoc. I’ve got a project that I haven’t quite figured out how to get on home brew (I’m new at this) that takes a file in <format> and lets me read it like a Man page. [2]

[1]: https://jblevins.org/projects/deft/ [2]: https://github.com/ashton314/homebrew-mm

[+] zaiste|6 years ago|reply
I went through many software solutions and ended up with Org Mode as well. There may be some clunkiness at the beginning and it definitely lacks the glamour of proprietary solutions such as Notion. But once you get a grip on Org Mode, you will be amazed how deep the rabbit hole goes. ;)

There is a great Org Mode tutorial [1] on YouTube by Rainer König.

The only « problem » is that you need to learn Emacs. I recommend Emacs Doom [3]

Also, I'm experimenting with making my Org Mode notes public. Here's a humble beginning [2]. I only publish them once they are cleaned, so it's just a tiny bit for now.

[1]: https://www.youtube.com/playlist?list=PLVtKhBrRV_ZkPnBtt_TD1... [2]: https://zaiste.net/programming/ [3]: https://github.com/hlissner/doom-emacs/

[+] gw666|6 years ago|reply
I am beginning to use a zettelkasten. I think it's an excellent method for doing deep thinking, especially over a long period of time. See https://www.seanlawson.net/2018/02/use-zettelkasten-method-s... and https://zoia.org/2018/11/13/zettelkasten/ for good introductions.

Reading this will change your life: https://www.amazon.com/How-Take-Smart-Notes-Nonfiction/dp/15...

I'm committed to text-only information. No matter how many notes I'm writing, I can always use grep or write my own programs to find what I want.

If you're on a Mac, The Archive is great (https://zettelkasten.de/the-archive/). It's what I'm using

zettelkasten.de has lots of good reading (thanks, Sascha and Christian!), but look to the other sources I've listed to get a more thorough idea of what a zettelkasten is (I call mine a thinking-box) and how to use one.

[+] sh87|6 years ago|reply
Last year, I went through this whole gamut of the usual suspects emacs(org-mode), plain text, markdown, tiddlywiki/ mediawiki/bunch of other wikis, google keep, one note, notational velocity, anki, evernote, LaTeX. One after the other, they all crumbled. I tried them all and failed. I got to a point of calm reflection based on my experience with each one. I realized what was most important for me. I eventually settled on a combination of email + git + google calendar. I use a separate email just for my notes. Every new thought, todo, bookmarks, quotes, images of doodles, event, information about people becomes a new email. Any update on the same one becomes a reply to the last email on that subject. Any topic or area that gathers enough content or insight that’s worth remembering goes to my git repo’s wiki page. If there’s enough content/files/code, it gets a directory. Common patterns, get linked via their respective wiki pages. I create collections of related topics as separate wiki pages with links back to the source.

This setup has worked well for me for over 6 months now. I do not worry about email / git becoming obsolete. Or how I would move my email/git data when I need to switch vendors. I can access it via any device I own or through someone else’s. Stuff I want to share with people are hosted via git pages or sent via email. I have a plethora of client applications to choose from. And above all... it just works.

I can give out more details but I feel anyone’s personal information needs can be met/exceeded with a combination of email + git + calendar.

[+] andrewsleigh|6 years ago|reply
Hi. I also thought of email when I was thinking about this problem recently. How do you get information from your email thread to the wiki? And is this a github wiki?
[+] lethologica|6 years ago|reply
This is a fascinating approach and would actually solve a lot of the problems I've had with trying this before. I would love to hear more if you had anything additional to add because this method has become the top of my list after trying several software packages mentioned thus far.
[+] pchr8|6 years ago|reply
I have something called "Diensttagebuch", which is awkward to translate ("Duty diary"?), it's a Jekyll blog (currently at https://www.pchr8.net/d/dtb/) for things I know I'll have to google later. I have two bash scripts for creating-or-opening a new page and for deploying (upload to server + commit to git). I use https://github.com/rking/ag.vim for searching though the Markdown files on my local machine.

For more general knowledge, Semantic Mediawiki (http://pchr8.net/f/), but it's basically a big link database now. I can sort it by rating / complexity / to read / domains / keywords and found it quite flexible.

Also, Anki for things I want to memorize. https://www.gwern.net/Spaced-repetition describes the theory quite well.

[+] iamnothere|6 years ago|reply
This is the most impressive example of the genre that I've seen: https://wiki.xxiivv.com/ (engine: https://wiki.xxiivv.com/#oscean)

For me, I use Orgmode, plus a mess of PDFs in folders on Google Drive. I've been looking into both Polar and Emacs as potential replacements for document organization.

Right now I have a single folder that I save new things to, and I have a recurring task to go back and file documents regularly. It's not perfect but it works. I don't often reference my pile of PDFs, though; I might use it more with tagging options, which is why I'm looking at replacements.

I used to use Evernote. There were some things I didn't like about it, although it's been so long that I couldn't tell you what they were anymore. It might be a good system now.

[+] mftrhu|6 years ago|reply
> This is the most impressive example of the genre that I've seen: https://wiki.xxiivv.com/ (engine: https://wiki.xxiivv.com/#oscean)

Yeah. I am currently using org-mode (plus org-wiki, some bespoke elisp and a few shell scripts) for my own knowledge base, but Oscean/the Nataniev ecosystem in general has impressed me.

plus Zotero for papers/PDFs, plus a pile'o'files stashed using the Universal Decimal Classification for its folder tree, plus assorted odds and ends.

In fact, I spent this week messing around with some of its concepts - Indental plus Runic plus a lightweight markup language loosely based on txt2tags + sexps and which should be easy to render even outside the browser.

The idea is to move away from documents (which org-mode is very good at), and towards smaller, rich chunks of information (which org-mode can't handle that well - I hacked things up with properties, but that namespace is flat).

[+] r_singh|6 years ago|reply
I use Evernote. I have multiple notebook stacks for different topics (like each company I work on, ideas, general life notes, learning, research, etc.) and within each stack have a notebook dedicated to notes addressing topics as well (for example in the company stack have a notebook for sales notes, in the learning stack have a notebook for JS).

Evernote has worked well so far and me trying to switch to OneNote didn't work out as well. I'm still open to alternatives, but Evernote has kept my mind organised without a doubt.

[+] GeorgeHahn|6 years ago|reply
I use BookStack (https://www.bookstackapp.com). MIT licensed & easy to self host.

It's fast and has good search, a good editor, and uses simple hierarchical organization + tagging. I like that it gets out of my way and does a good job of handling everything from code snippets to freeform thoughts to recipes.

BookStack is best for things that 'look like' documents. Mobile editing could also be better - you have to click a maximize button to get a usable view of the editor and the editor UI still takes up a lot of space. It is web based, but it is very lightweight and I have found it very easy to host.

[+] jandeboevrie|6 years ago|reply
Both public and private. Public, a static site made from markdown files : https://raymii.org. Almost 400 articles now.

private, a folder called "notes", with almost a thousand short text files. Workflow for it? Use grep to find what I want. I had a system of folders per topic but that became a mess. Grep is good enough. The notes folder is synced with synching to multiple systems and my phone. On the phone I have termux, which has grep and markor as text editor.

[+] pjmorris|6 years ago|reply
I used to use FreeMind [0], a mind-mapping tool that I really like, to keep track of projects, ideas, and things to follow up on. I eventually noticed that the pretty pictures could be represented, without loss of information, by a simple outline in a text file. Reading 'Getting Things Done' [1] got me to think in terms of a 'Current Task' list, which I now keep as a text file, backed up on Dropbox. Anything I need to do, or want to follow up on, or want to dream about in a practical way, gets jotted down in the file, with the most urgent matters at the top, Some really interesting stuff drifts down in the file as time passes.

As I do stuff, things move from the Current Tasks file to a Logbook file. Beyond 'accomplished todo's, that file gets stuffed with, for example, interesting quotes I run into, command lines I think I'll need again, and a record of what I've been up to. When I need to put together a status report, or figure out for myself where my time went, or recall how to do something intricate that I haven't done in six months, I turn to my Logbook.

With these two pieces, I feel like I've got a handle on what I want to do, and a record of what I've done. Now, if I just could find the time to follow through...

[0] http://freemind.sourceforge.net/wiki/index.php/Main_Page

[1] https://en.wikipedia.org/wiki/Getting_Things_Done

[+] kd5bjo|6 years ago|reply
I started with the augmentingcognition.com spaced repetition model, and then decided to augment it with a catalog of everything I read(1). For my own original thoughts, I write them down in a few paragraphs and then treat them like any other reading material.

At some point, I ran across the idea that a library card catalog is really just a combined topical index for the entire collection, with each entry being a card. That seemed like a decent model, and I’ve been happy with it so far.

My system is currently a physical card index because I needed the flexibility of pen & paper to figure out how I wanted it to work, but I’ve been slowly computerizing it as I figure things out- you can certainly do something similar with flat text files, spreadsheets, or any other general purpose document software.

So now, I have three interacting systems: a citation list that records where I can find things to read them again, a combined topical index of everything I’ve read, and a flash card deck of things I want to remember without having to look them up.

When I run across an interesting article on the web, for instance, I save a PDF of it to Dropbox for safe keeping and print a copy to read away from my computer. As I read, I’ll make margin notes and have two different colors of highlighter: one to note index headings and another to mark passages to be turned into flash cards.

After my morning flash card review, I go through the material I read the previous day and turn all of the notes and highlights into cards(2). Once everything has been moved into the indexing system, I have no further need for the physical copy, so it goes to the recycler.

(1) Of sufficient length; generally 3 or more printed pages.

(2) Initially by hand, now with the aid of a custom piece of software

[+] zhte415|6 years ago|reply
Yes I do but I wish I didn't keep it so haphazard.

By haphazard I mean to focus not only on acquisition but reflection / contemplation: Collecting thoughts is one thing, putting them together into a cohesive and coherent body of knowledge is another. So, that's what I'm working on.

pmwiki for now. Workflow is personal domain, putting browser or window on one side of the screen and what I'm reviewing, or what I'm thinking / text editor (sublime text) of the other side. Putting into a DIY solution based on flask simply because that's flask practice.

I also like pens and paper and whiteboards. So take pictures and use Powerpoint's 'remove background' feature to have a liberal amount of pictures.

A bit rambling, but you asked and asking is usually best! So.. :D !

[+] rmkrmk|6 years ago|reply
A bunch of markdown/text files in a git repo, hosted on github/gitlab/gitea (github in my case).

This way I can edit the files online on github.com, on my desktop and push/pull the changes. On my iOS device I use http://workingcopyapp.com - I can edit and create files as well as push and pull changes.

On my Desktop I can use any tool I want to search and edit the files, on github I have full text search and working copy also features full text search.

Most importantly: I own my data and they don't up in some database I need to worry about.

[+] Chirael|6 years ago|reply
I use OneNote... just kidding, I actually use TiddlyWiki
[+] samrohn|6 years ago|reply
I use google colab (https://colab.research.google.com/notebooks/)for things like work notes on setting things up, any new tech I am learning. The best thing is it support code(python). Google keep(http://keep.google.com) for noting down any interesting thoughts/ideas/lesson I learn through out the day. I usually create a card per month to keep it organised.
[+] mark_l_watson|6 years ago|reply
Really interesting idea to use Colab. I assume that you occasionally download the notebooks for local backup? Also, are downloaded notebooks useable with Jupiter notebook?
[+] codazoda|6 years ago|reply
Lately I've just been using a directory full of markdown files. I store my personal collection in Dropbox but it would be just as easy to use git or any other tool. I've created an alias to automatically open that directory in my favorite editing tool when I type "notes".

I also manually maintain an index.md file to categorize things, but it's not really necessary when you can just use grep or your editors search feature to find what you need.

[+] barryhoodlum|6 years ago|reply
I do the exact same, down to the "notes" alias. I find the downside to this approach is it doesn't work well on mobile - the Dropbox app can edit files but there's no search and it all feels a bit clunky.