Why would you store all of this valuable information in a proprietary tool? What happens when notion shuts down, jacks up prices, makes feature changes you don't like, etc etc? Make some attempt to migrate all this data? It's gonna be hard to do this in an automated way, if they even have an (export) api for this.
I'm using (vim)wiki/gollum and I feel much safer about my data.
Obviously, you have a point. I am indeed a little bit worried.
Some things that ease my mind:
Notion's export and backup features are quite good. You can export everything in various formats, markdown being the most relevant for me.
This only becomes messy with complex databases that use formulas and are related to each other. You would not have these with local solutions in the first place.
Notion is already profitable and growing rapidly, I do not see them shutting down in the foreseeable future.
However, the other concerns are relevant.
The API is supposed to be released soon.
I intend to either build a backup workflow myself or use other tools that will get developed then.
Fully agree. I was turned off notion for the same reason and reverted back to self-hosting dokuwiki. A much more limited feature set, to be sure, but at least I'll still have my notes 10 years from now.
Does anyone else use plain text notes for everything?
I've tried to optimize things so that while working, I can get things jotted down with close to zero resistance. If the barrier of entry is too high then I don't bother doing it.
That means often being able to spawn a new terminal, write a command and be done with it. Or to be able to pipe something from my clipboard into an auto-dated file.
I'm experimenting with a single 500K+ line-long flat text file for EVERYTHING, from dream journalling to work email drafts to blog posts etc. -- I've even copied math from my home whiteboard in latex -- screw rendering, I'll transcribe them back when I need them.
At one point I was trying to do a Luhmann's Zettelkasten with one note per txt file, internal linking, etc. I concatenated every piece of txt content I had. It's much simpler.
I have no system anymore. I try to create internal "hashtags" that are ctrl-F searchable. The goal is having something that's always open and at hand. I got plugins for autosaving when the window loses focus, autoloading when changed in disk and it syncs to Dropbox.
I cobbled together in Notepad++ (which has a GUI for this) a small syntax coloring definition that sort of matches my spontaneous habits from old salvaged text. I intend to let this evolve too. E.g. I'm using {braces} to have a little collapse button that hides pieces of text.
Tangentially related, but after years of trying out (and struggling with) hundreds of calendar and schedule apps, in the end what was best for me turned out to be a simple text file that I keep updating and look at each day. A line on it represents a day and begins with (dd/mm), and events for the day are plaintext separated by commas. Each day gone by gets erased. Absolutely zero friction and I can see an entire month of stuff at a glance so I'm always aware of what to do. Can't believe it took me so long to go back to such bare basics, and now I strive to reach this level of brutal basic frictionless simplicity in my own toy projects.
This is how I use Sublime Text cause I dont have to actually save new files it saves them for me. It loads instantly so its perfect. I paste in code snippets and what have you.
I dont use ST3 for proper dev as much cause I use JetBrains IDEs more.
I still would prefer something better for my more permanent notes.
Yep, zero resistance: plaintext, with context, metadata, keywords, minimal formatting, grep and a few scripts. The most important thing to remember is that I don't remember everything, so write it down, and use sensible filenames/hierarchy/tags/metadata so it can be found easily.
The less temptation to play with CSS, formatting and markup the better. One day it'll all go in a self-contained minimal wiki. Maybe.
On my android phone I use an ancient app called "unote" (installed from an .apk which I keep, it's long since disappeared from Play). Simple, hierarchical plaintext notes, kept in a local sqlite file. Sadly no search, but that just means I have to be disciplined about what, where and pruning.
I use emacs org mode which stores every thing in a text file.The text file is fully human readable even without emacs. Its has few rules that makes the collapsing the parts of the text file. One of the main rule is any line that starts with asterik followed by space ("* ") is interpreted as heading and the whole heading can be collapsed.
Yes; I've been using Notational Velocity for years. It gives just enough automatic structure (it's almost like a box filled with index cards, though the size of the note is unlimited) without having to actually fiddle with files. It's effortless to create new entries and incredibly easy to find things again later.
Academic papers, standards documents, white papers etc... get downloaded as PDF and renamed according to a (human-readable) standard scheme that I use (year, title, [author(s)], [paper-type]), and stored in a 'meaningful' filesystem hierarchy. Podcasts and some videos are also stored in the same hierarchy.
Documentation for the software libraries and APIs that I use is downloaded from readthedocs (where possible) and stored in a parallel system that takes account of versioning. (So I can concurrently store differently versioned copies of the documentation for a single library).
I have a simple python script that iterates through my directory hierarchy and produces a sqlite database and a couple of xlsx files with a row-per document (one spreadsheet for reporting document-management metadata and another that allows me to assign labels and write precis notes). The script also extracts the content of the PDFs as plain-text and feeds some NLP tools that I'm playing with.
I use the spreadsheets to keep notes on the files, and the act of manually renaming and sorting the PDFs into the 'right' place in the hierarchy helps me to understand what's in them and remember what I've got. (I'm constantly reorganising the hierarchy as my understanding develops and evolves. My python script keeps everything -- notes and documents and other metadata - in sync).
What software do you use to rename PDF files and extract their metadata automatically? I have found this difficult to accomplish, despite trying multiple different tools. PDFs from Arxiv are commonly a problem.
I’d like to look into the features of notion more, but I can’t help but wonder why it isn’t more common to see people using a local SQLite database, or any other self hosted database, to serve as a personal knowledge base management system. A lot of the paid solutions I’ve seen seem to bend over backwards to offer a limited subset of the features that are trivially available in an actual database.
Has anyone tried out using a personal database like this?
For managing a personal knowledge base, I highly recommend Tiddly Wiki[1]
It's a self-contained Wiki/Notebook/Journal with tags, it works instantly, and all of it is in a single HTML file with magic JS in it. Or you can use it with a server.
I use a tool called zim wiki. Its less of a wiki and more of a notes app with a tree sidebar for navigation and linking between pages. Everything is stored as txt files which works awesome.
I would love an alternative that is free/and or possible to self-host.
However, I do not see how this compares to Notion. How do I edit this HTML file on my phone? How do I add bookmarks to it via a hotkey? What is the equivalent of Notion's databases? Does it have Slash commands? This list of questions could go on for much longer. I really don't want to sound like a Notion shill, especially since I also use other tools (e.g. GitLab) for my knowledge base, but many of the suggested alternatives lack basically all the features that I describe in my post.
How do you recommend running it? The install guide lists a half-dozen different possibilities including PHP, Node, Ruby which all look to be vastly different solutions.
I strongly support this recommendation. Tiddly Wiki has enough features and extensibility that I wasn’t tempted to write my own bibliography management system after discovering it. Yes, I know there is dedicated software for that but they didn’t fit how I want to structure my notes.
For Links/Bookmarks I use Google bookmarks - https://www.google.com/bookmarks/
which surprisingly has not been killed and looks like might be left alone by Google.
Other bookmarking services delicious.com & posterous etc all died in due time.
By using browser extensions and a 3rd party Mobile App. I am able sync/store all the links. Though I would love to be able to sync bookmarks to my org-mode notes. Looking at org-capture now to see if that could possibly work (https://github.com/alphapapa/org-protocol-capture-html)
It’s all I need, I have tens of thousands of lines of notes at this point, can search them (text, regexes, tags, whatever) essentially instantly, can export to html or slides or pdf or $format, will never go down or close down or disappear behind a paywall, it’s perfect
I also love org-mode, but I have to admit that asciidoc and asciidoctor has been growing on me, especially for anything I want others to contribute to.
For this I use Notable[0] with its data directory located in a GitLab repo. A combination of Markdown and Git with Notable's impressive UI. You can also use any cloud drive, Cryptomator vault, etc. for the storage backend.
Notable's categories (tags) for me include things like manpage snippets, code snippets, workspaces/scratchpads, thoughts, etc.
Of note, a lot of the things in my head that I need to jot down on my mobile, even at length, are usually stored in Simplenote[1] then transferred to Notable if they're important enough, via Simplenote's desktop app.
What I would love is a full personal CMS type system, where I can have a subject like "cybersecurity" or "cooking" and I can add bookmarks, notes, pdfs, and they're all treated equally and as first class objects in the system.
I would love a system that treats all media the same as just a source of knowledge instead of being hung up on the source type i.e. is it a video, or a book mark or a pdf....
I've been using Zim [1] as a knowledge database for around 2 years. I don't like depending on online solutions which may suddenly disappear.
Together with some plugins for managing tasks, git and some script for synchronizing repositories, it has been working great.
There are some limitations on this approach (ie, two persons editing the same page concurrently is a no-no), but for my use-case it works perfectly.
I find it much more intuitive than org-mode, and the fact that it auto-generates a "global" task list based on items spread across the whole notebook makes it much easier to prioritize things.
My note app hopping days ended the moment I first encountered Joplin, which I now use for almost all my writing.
Killer feature being the encrypted sync via more or less any kind of storage back end (where presently I use Box via WebDav, but that's easily switched). Seemless work across all my devices, all data in nice and portable markdown.
It fits my needs so well that I have taken the extraordinary step of allowing the desktop version on my pc's, even though it is Electron based.
I have a single God ReadMe file in which I try to record all the facts that I should know and 'recipes' that I might need to follow again. Such as that shutting down windows does not dispose of kernel state whereas restart does. Or how to do a certain task on the income tax site or how to set up an odbc datasource in Windows. I've been keeping this file for years and I periodically read through it as much to remind myself of what's in it as anything else.
It's saved me a lot of frustration when finding myself having to do things again that I know it took me ages to figure out the previous time but I have since forgotten.
A tool is great for collaboration, but how much do you really need for a personal KB? And how much effort do you want to spend migrating?
Then there's vendor lock-in, learning curves, etc.
A well-organized directory structure with plain text files gives you a data store that won't become obsolete and can be used on any OS. I use CLI tools to search (e.g., grep) and that's all the features I need right there.
I've mentioned this before, but I'd love to have a note taking app as convenient as EverNote, but that lets me insert code cells a la Jupyter notebook. I can barely think without Jupyter any more. Maybe that's not a good thing, but it's sure handy.
Try Joplin, its a direct replacement for EverNote, can import its database, and has active development to add more similar function. The only one that I'm missing right now is OCR, not sure if its coming, though. I'm using dropbox to sync my phone with laptop clients.
Hmm, what is it you don't like about code blocks in Notion? Don't think it can get much more convenient. The only things I miss is auto-format and maybe automated language detection. I don't know of any web-based tool that can do those things though, except some that do only those things.
This has been a workflow that served me well for the last couple of years -
Most of my stuff is on dropbox/google drive for accessing them across different devices. I prefer tools that let me control where the data is and not the other way around.
For all academic papers, documents, I use Zotero. you can use your favorite pdf reader to annotate or take notes on pdf's and zotero will sync these. I also love the feature where Zotero can automatically extract all annotations from the pdf. (I some times save these as an org file)
If I am reading a longform web article or a blog post that I really think is useful and helpful, I also save these to Zotero. The push to kindle extension from fivefilters is an awesome tool that converts webpages to pdf.
I'm currently testing the memex extension from worldbrain to annotate and organize my browsing history.
For all notes, journals, random thoughts, ideas, (both work and personal), I use orgmode. (I recently switched from ZimWiki). Its been amazing so far. So many things are easier to do orgmode although the learning curve for emacs is pretty steep in the beginning.
On mobile, I use the orgzly app for accessing and taking notes! Its by far the best android app I use so far.
I also found https://dabble.me/ the other day which seemed like a nice slot in for quick thoughts with the ability to email to your journal. I've heard that journaling manually works better but I really dislike writing unless necessary.
One of my biggest gripes is with how many places I have my data in currently, and as a dev, I should have done something already but that's another story.
I'll definitely have a look at Notion and org-mode having read the feedback here along with a few other mentions elsewhere.
I was a paying EverNote user for years, but they stopped evolving while other NoteTaking apps didn't. I forget which feature they removed that was the last straw, but the recipe front-end was hugely important to me.
The problem with bookmarks is you're never going to have time to reread the whole thing. I make a summary and put the source as a hyperlink so that if I summarized wrong I can go read it again. Summaries are better because you can organize summaries and have everything you know about enums together in a few pages, rather than 20 bookmarks.
I'm a bit surprised nobody seem to have mentioned Bear Notes.
I've used Onenote, Org Mode in Emacs but I am very happy to have discovered Bear Notes (https://bear.app/) last. year. It has really changed how much I document, work with text, store info etc.
Bear handles images, lists etc inline. It allows for very easy tagging (creating hierarchies) etc. And it looks really nice. Even with good fonts, Org Mode never really looked pleasing to me [0]. And I found that easthetics is really important for me to actually use the tool.
Bear uses SQLite for storage, which can be accessed with any SQLite tool (I've tested it). And it can also do batch export to a number or formats. The exported files looks good and can easily be imported into other tools, for example Org Mode (I've tested it). Bear can also export to PDF with styles. I actually use this to export my notes as reports to the company board, customers etc. And I've gotten compliments on how good the reports looks. ;-)
Finally Bear syncs easily. I can now work with my notes on the phone, iPad and laptop.
If I only could get the tool to use a solid, non-blinking caret life would be a bliss. Go Bear!
[+] [-] Xunxi|6 years ago|reply
This video covers some of the things you can do with it in an academic setting but relatable. https://m.youtube.com/watch?v=JQD5c8A_D2g
0:33 - Math equations
1:45 - Replay text
2:45 - Ink to text
3:20 - Research tools
4:39 - Immersive reader
6:01 - Web clipper browser extension
7:12 - Save emails to OneNote
NB: https://www.microsoft.com/en-us/microsoft-365/blog/2015/11/1...
[+] [-] travolter|6 years ago|reply
[+] [-] tkainrad|6 years ago|reply
Some things that ease my mind:
Notion's export and backup features are quite good. You can export everything in various formats, markdown being the most relevant for me. This only becomes messy with complex databases that use formulas and are related to each other. You would not have these with local solutions in the first place.
Notion is already profitable and growing rapidly, I do not see them shutting down in the foreseeable future. However, the other concerns are relevant.
The API is supposed to be released soon. I intend to either build a backup workflow myself or use other tools that will get developed then.
[+] [-] henriks|6 years ago|reply
[+] [-] nickjj|6 years ago|reply
I've tried to optimize things so that while working, I can get things jotted down with close to zero resistance. If the barrier of entry is too high then I don't bother doing it.
That means often being able to spawn a new terminal, write a command and be done with it. Or to be able to pipe something from my clipboard into an auto-dated file.
I ended up putting together this ~15 line Bash script which seems to do the trick: https://github.com/nickjj/notes
I've been using plain text notes since 2001, although I only recently created this script. So far so good.
[+] [-] thanatropism|6 years ago|reply
At one point I was trying to do a Luhmann's Zettelkasten with one note per txt file, internal linking, etc. I concatenated every piece of txt content I had. It's much simpler.
I have no system anymore. I try to create internal "hashtags" that are ctrl-F searchable. The goal is having something that's always open and at hand. I got plugins for autosaving when the window loses focus, autoloading when changed in disk and it syncs to Dropbox.
I cobbled together in Notepad++ (which has a GUI for this) a small syntax coloring definition that sort of matches my spontaneous habits from old salvaged text. I intend to let this evolve too. E.g. I'm using {braces} to have a little collapse button that hides pieces of text.
[+] [-] phreack|6 years ago|reply
[+] [-] giancarlostoro|6 years ago|reply
I dont use ST3 for proper dev as much cause I use JetBrains IDEs more.
I still would prefer something better for my more permanent notes.
[+] [-] mrspuratic|6 years ago|reply
The less temptation to play with CSS, formatting and markup the better. One day it'll all go in a self-contained minimal wiki. Maybe.
On my android phone I use an ancient app called "unote" (installed from an .apk which I keep, it's long since disappeared from Play). Simple, hierarchical plaintext notes, kept in a local sqlite file. Sadly no search, but that just means I have to be disciplined about what, where and pruning.
[+] [-] thomas232233|6 years ago|reply
[+] [-] wool_gather|6 years ago|reply
[+] [-] aqw137|6 years ago|reply
https://github.com/panos-zamos/zlog-cli#lt
[+] [-] jayaram|6 years ago|reply
[+] [-] w_t_payne|6 years ago|reply
Academic papers, standards documents, white papers etc... get downloaded as PDF and renamed according to a (human-readable) standard scheme that I use (year, title, [author(s)], [paper-type]), and stored in a 'meaningful' filesystem hierarchy. Podcasts and some videos are also stored in the same hierarchy.
Documentation for the software libraries and APIs that I use is downloaded from readthedocs (where possible) and stored in a parallel system that takes account of versioning. (So I can concurrently store differently versioned copies of the documentation for a single library).
I have a simple python script that iterates through my directory hierarchy and produces a sqlite database and a couple of xlsx files with a row-per document (one spreadsheet for reporting document-management metadata and another that allows me to assign labels and write precis notes). The script also extracts the content of the PDFs as plain-text and feeds some NLP tools that I'm playing with.
I use the spreadsheets to keep notes on the files, and the act of manually renaming and sorting the PDFs into the 'right' place in the hierarchy helps me to understand what's in them and remember what I've got. (I'm constantly reorganising the hierarchy as my understanding develops and evolves. My python script keeps everything -- notes and documents and other metadata - in sync).
So far this has scaled OK to around 26,000 PDFs.
[+] [-] HSO|6 years ago|reply
Ps. Another question: can you tell which nlp libs are best for this purpose in your experience (and how do you eventually search the generated index?)
[+] [-] j15t|6 years ago|reply
[+] [-] bnj|6 years ago|reply
Has anyone tried out using a personal database like this?
[+] [-] romwell|6 years ago|reply
It's a self-contained Wiki/Notebook/Journal with tags, it works instantly, and all of it is in a single HTML file with magic JS in it. Or you can use it with a server.
[1]https://tiddlywiki.com/
[+] [-] Polylactic_acid|6 years ago|reply
[+] [-] tkainrad|6 years ago|reply
However, I do not see how this compares to Notion. How do I edit this HTML file on my phone? How do I add bookmarks to it via a hotkey? What is the equivalent of Notion's databases? Does it have Slash commands? This list of questions could go on for much longer. I really don't want to sound like a Notion shill, especially since I also use other tools (e.g. GitLab) for my knowledge base, but many of the suggested alternatives lack basically all the features that I describe in my post.
[+] [-] whalesalad|6 years ago|reply
[+] [-] lwhsiao|6 years ago|reply
[+] [-] tasogare|6 years ago|reply
[+] [-] throwaway_2047|6 years ago|reply
[+] [-] antjanus|6 years ago|reply
I have a single file called "engineering-notebook" and I store pretty much everything in it:
- bookmarks to resources
- personal notes
- personal tutorials/step-by-steps
- cheatsheets
It works fantastically for me
[+] [-] nausher81|6 years ago|reply
For Links/Bookmarks I use Google bookmarks - https://www.google.com/bookmarks/ which surprisingly has not been killed and looks like might be left alone by Google. Other bookmarking services delicious.com & posterous etc all died in due time.
By using browser extensions and a 3rd party Mobile App. I am able sync/store all the links. Though I would love to be able to sync bookmarks to my org-mode notes. Looking at org-capture now to see if that could possibly work (https://github.com/alphapapa/org-protocol-capture-html)
Chrome Extension - https://play.google.com/store/apps/details?id=com.amlegate.g...
Firefox extension - https://addons.mozilla.org/ru/firefox/addon/fess-google-book...
Android App for Google Bookmarks (3rd Party) https://play.google.com/store/apps/details?id=com.amlegate.g...
[+] [-] sandinmyjoints|6 years ago|reply
[+] [-] bachmeier|6 years ago|reply
[+] [-] arminiusreturns|6 years ago|reply
TODO alone is a game changer imho.
[+] [-] The_Colonel|6 years ago|reply
[+] [-] oedmarap|6 years ago|reply
Notable's categories (tags) for me include things like manpage snippets, code snippets, workspaces/scratchpads, thoughts, etc.
Of note, a lot of the things in my head that I need to jot down on my mobile, even at length, are usually stored in Simplenote[1] then transferred to Notable if they're important enough, via Simplenote's desktop app.
[0] https://notable.md [1] https://simplenote.com
[+] [-] nickthemagicman|6 years ago|reply
I would love a system that treats all media the same as just a source of knowledge instead of being hung up on the source type i.e. is it a video, or a book mark or a pdf....
[+] [-] abacate|6 years ago|reply
Together with some plugins for managing tasks, git and some script for synchronizing repositories, it has been working great.
There are some limitations on this approach (ie, two persons editing the same page concurrently is a no-no), but for my use-case it works perfectly.
I find it much more intuitive than org-mode, and the fact that it auto-generates a "global" task list based on items spread across the whole notebook makes it much easier to prioritize things.
[1] https://zim-wiki.org/
[+] [-] blurbleblurble|6 years ago|reply
I'm hoping (and optimistic) that the distributed web brings with it some epiphanies about how to do better local knowledge management!
[+] [-] jhoechtl|6 years ago|reply
[+] [-] interfixus|6 years ago|reply
Killer feature being the encrypted sync via more or less any kind of storage back end (where presently I use Box via WebDav, but that's easily switched). Seemless work across all my devices, all data in nice and portable markdown.
It fits my needs so well that I have taken the extraordinary step of allowing the desktop version on my pc's, even though it is Electron based.
https://joplinapp.org/
[+] [-] tenant|6 years ago|reply
[+] [-] brushfoot|6 years ago|reply
A tool is great for collaboration, but how much do you really need for a personal KB? And how much effort do you want to spend migrating?
Then there's vendor lock-in, learning curves, etc.
A well-organized directory structure with plain text files gives you a data store that won't become obsolete and can be used on any OS. I use CLI tools to search (e.g., grep) and that's all the features I need right there.
[+] [-] analog31|6 years ago|reply
[+] [-] hohloma|6 years ago|reply
https://joplinapp.org/
[+] [-] theonething|6 years ago|reply
[+] [-] tkainrad|6 years ago|reply
[+] [-] jayaram|6 years ago|reply
Most of my stuff is on dropbox/google drive for accessing them across different devices. I prefer tools that let me control where the data is and not the other way around.
For all academic papers, documents, I use Zotero. you can use your favorite pdf reader to annotate or take notes on pdf's and zotero will sync these. I also love the feature where Zotero can automatically extract all annotations from the pdf. (I some times save these as an org file)
If I am reading a longform web article or a blog post that I really think is useful and helpful, I also save these to Zotero. The push to kindle extension from fivefilters is an awesome tool that converts webpages to pdf.
I'm currently testing the memex extension from worldbrain to annotate and organize my browsing history.
For all notes, journals, random thoughts, ideas, (both work and personal), I use orgmode. (I recently switched from ZimWiki). Its been amazing so far. So many things are easier to do orgmode although the learning curve for emacs is pretty steep in the beginning.
On mobile, I use the orgzly app for accessing and taking notes! Its by far the best android app I use so far.
practices >> tools!
[+] [-] dross|6 years ago|reply
I also found https://dabble.me/ the other day which seemed like a nice slot in for quick thoughts with the ability to email to your journal. I've heard that journaling manually works better but I really dislike writing unless necessary.
One of my biggest gripes is with how many places I have my data in currently, and as a dev, I should have done something already but that's another story.
I'll definitely have a look at Notion and org-mode having read the feedback here along with a few other mentions elsewhere.
[+] [-] Kihashi|6 years ago|reply
[+] [-] metachris|6 years ago|reply
[+] [-] heavenlyblue|6 years ago|reply
[+] [-] Noumenon72|6 years ago|reply
[+] [-] JoachimS|6 years ago|reply
I've used Onenote, Org Mode in Emacs but I am very happy to have discovered Bear Notes (https://bear.app/) last. year. It has really changed how much I document, work with text, store info etc.
Bear handles images, lists etc inline. It allows for very easy tagging (creating hierarchies) etc. And it looks really nice. Even with good fonts, Org Mode never really looked pleasing to me [0]. And I found that easthetics is really important for me to actually use the tool.
Bear uses SQLite for storage, which can be accessed with any SQLite tool (I've tested it). And it can also do batch export to a number or formats. The exported files looks good and can easily be imported into other tools, for example Org Mode (I've tested it). Bear can also export to PDF with styles. I actually use this to export my notes as reports to the company board, customers etc. And I've gotten compliments on how good the reports looks. ;-)
Finally Bear syncs easily. I can now work with my notes on the phone, iPad and laptop.
If I only could get the tool to use a solid, non-blinking caret life would be a bliss. Go Bear!
[+] [-] benhurmarcel|6 years ago|reply