Ask HN: How do you organize document digests / personal knowledge?
8 comments
I'm the author of Polar - https://getpolarized.io/
Some of this functionality will land in the short term. We're working on adding the following:
- a sidebar for documents with extended metadata (publication year, authors, abstract, etc)
- The ability for notes directly on the document itself. This would give you your 'overview'. Lots of people want to use it as a summary feature. We might actually add a summary field.
- links between the documents and directly to annotations.
... in the longer term we're going to add more complicated full-text search of the documents themselves. My background is search so I want to do this right using Elasticsearch as the backend so that we can add more advanced features like clustering, auto-tagging, etc.
Search is probaby 90 days away I hope. The other features are on the short term. I might actually flag the summary feature a bit higher as a a lot of people are asking for it and it's pretty easy to add technically but I need to work on some UI around the summaries.
Maybe a non-perfect UI is good to get us off the ground.
Oh. I'm also working on adding metadata automatically from documents you upload like author, publication year, etc.
We do about 2x releases a month so dev time is iterating really fast!
If you'd like us to go faster buy premium ;)
Some of this functionality will land in the short term. We're working on adding the following:
- a sidebar for documents with extended metadata (publication year, authors, abstract, etc)
- The ability for notes directly on the document itself. This would give you your 'overview'. Lots of people want to use it as a summary feature. We might actually add a summary field.
- links between the documents and directly to annotations.
... in the longer term we're going to add more complicated full-text search of the documents themselves. My background is search so I want to do this right using Elasticsearch as the backend so that we can add more advanced features like clustering, auto-tagging, etc.
Search is probaby 90 days away I hope. The other features are on the short term. I might actually flag the summary feature a bit higher as a a lot of people are asking for it and it's pretty easy to add technically but I need to work on some UI around the summaries.
Maybe a non-perfect UI is good to get us off the ground.
Oh. I'm also working on adding metadata automatically from documents you upload like author, publication year, etc.
We do about 2x releases a month so dev time is iterating really fast!
If you'd like us to go faster buy premium ;)
I am/was faced with similar both in my professional and personal projects. The only tool that I could aggregate everything I need to reference in it's native format was Evernote. While it is not perfect (def. not) it has been able to meet everything I need...the rest was developing a structure to tag, index, and funnel everything to something useful...I'm light years ahead now, but still polishing the terd shall we say....I hope maybe some day there will be a cross between Evernote and the power of Jupiter-like notebooks.
Have you written a blog post about how you handle this by any chance? I would be interested in learning more about it.
I am currently writing an extensive blog post on this topic, still several weeks away from being finished though.
I use several tools, but mainly Notion - https://www.notion.so/. It can do everything you ask, except its handling of PDF documents is not very sophisticated (e.g. no OCR) and I am not too excited about their web clipper.
Everything else would be a great fit for you. You could create tables with your documents, allowing you to label them, set whatever properties you like and easily cross-reference them within your notes.
I use several tools, but mainly Notion - https://www.notion.so/. It can do everything you ask, except its handling of PDF documents is not very sophisticated (e.g. no OCR) and I am not too excited about their web clipper.
Everything else would be a great fit for you. You could create tables with your documents, allowing you to label them, set whatever properties you like and easily cross-reference them within your notes.
It doesn't seem to be able to annotate embedded PDF's which is a deal-breaker for me.
I tried notion but unfortunately, gave up after 5 mins (Android). Pressing the phone's back button closes the app, and the options to confirm changes ("done" button) are on the top, making me have to reach for the top of the screen all the time... :(
I think the app has gotten a lot better recently. I am not so sure what you mean by confirming changes/done button - Usually things are saved directly without confirmation. However, I do agree that Notion is best used with a laptop/desktop. Many of the more sophisticated features are too cumbersome on mobile.
In my opinion the best way to do it, albeit quite time consuming to setup, is to setup orgmode with extensions such as org-ref, pdf-tools and org-brain. It is a big hassle, however I see it as do-once-in-a-lifetime (and iterate till infinity) kind of setup.
The possibilities are virtually limitless, the only minor (depending on what you want) setback is that the notes are "plaintextish" and separated from pdf (I see that as benefit but YMMV). There are interlinks between the text and org files, but the comments are not "in" the pdfs if that's what you looking for.
I can't provide any links at the moment but you will find many examples looking at the blogs. Be aware org is borg and consumes all information in your life ;). Fortunately it's as local or cloud and as encrypted as you make it.
The possibilities are virtually limitless, the only minor (depending on what you want) setback is that the notes are "plaintextish" and separated from pdf (I see that as benefit but YMMV). There are interlinks between the text and org files, but the comments are not "in" the pdfs if that's what you looking for.
I can't provide any links at the moment but you will find many examples looking at the blogs. Be aware org is borg and consumes all information in your life ;). Fortunately it's as local or cloud and as encrypted as you make it.
I like using Org, but without adding too many extensions. More like a Zettelkasten.
I use [1](https://www.devontechnologies.com/apps/devonthink) Devonthink does all you need AND is very resourceful.
I do send articles from the Safari to Devonthink, and later, I check the "Inbox" (there are several ways to make automatic rules) Once there, I tag it.
My groups have a smart selection by tags.
And It has an RSS reader and can archive web pages.
Easy, Cloud support, Encryption support, and multi-database.
Disclaimer: I am using it for two years on my desktop and laptop (syncing by iCloud), and I don't earn anything selling it.
I do send articles from the Safari to Devonthink, and later, I check the "Inbox" (there are several ways to make automatic rules) Once there, I tag it.
My groups have a smart selection by tags.
And It has an RSS reader and can archive web pages.
Easy, Cloud support, Encryption support, and multi-database.
Disclaimer: I am using it for two years on my desktop and laptop (syncing by iCloud), and I don't earn anything selling it.
Doesn't support Windows/Linux and here is the response from the support (Jim Neumann, Customer Relations Specialist):
> Sorry, but no. We are Apple only developers. Also the underlying technical structure of the database should not be messed with. The best bet: Buy a Mac and DEVONthink 3 Pro. You'll feel better about life. I promise. Haha!
> Sorry, but no. We are Apple only developers. Also the underlying technical structure of the database should not be messed with. The best bet: Buy a Mac and DEVONthink 3 Pro. You'll feel better about life. I promise. Haha!
Awesome solution, spend a couple thousand dollars and change OS :D
This is going to sound old school, but I keep things nicely organized with OneNote2016. (A version that still allows you to keep the files on Dropbox, rather than having to deal with the inferior MS cloud.)
What I like is the fact that there is intuitive structure, with (project) notebooks, folders and pages. That makes it easy to add external content and then make notes, although for true annotation, you probably want to manually copy webpages into OneNote, rather than the PDF-style print to OneNote.
What I like is the fact that there is intuitive structure, with (project) notebooks, folders and pages. That makes it easy to add external content and then make notes, although for true annotation, you probably want to manually copy webpages into OneNote, rather than the PDF-style print to OneNote.
I started using Joplin recently - https://joplinapp.org/ - it has a browser plugin. It can store things locally and then you can use the search engine to look up your bookmarks, together with your notes, tags.
for Mac i still use Outline (like OneNote)
I automatically OCR PDFs and downloaded images to make them searchable.
==> Do you know a good tool that would support my workflow?
──────────────────────────────────────────
Specifically, here is the main workflow that I'm trying to support:
– Import documents easily (PDF, web pages), bonus point if I can easily reference the source URL.
– Annotate those documents (annotations of highlights like Polar). Write a quick 1-2 line summary about what it deals with (+keywords?).
– Write a digest of the document (several paragraphs). I want to be able to read a document, summarize the most interesting parts and reference the document. Important: inline images in the digest because documents can contain useful illustrations.
– Write a summary about a subject, referencing multiple digests/documents.
– Full search over all my notes/stored documents. Give priority to my notes. (word2vec-style embedding for keywords search would be an amazing addition but I don't count on it).
──────────────────────────────────────────
Tools that I've considered:
– GitHub Gist/Git with markdown files: I use this currently (one Gist for each subject I care about). GitHub search is lackluster, imprecise doc cross-ref.
– Polar: I use this to annotate documents. Doesn't allow me to write digests about the documents I read.
– Zotero: Doesn't support annotations anymore[4].
– Mendeley: Proprietary and encrypted database[5]. I want my knowledge repo to be accessible in 10 years.
– Bookstack: Does it support cross-referencing to documents?
[1] https://news.ycombinator.com/item?id=17892731
[2] https://news.ycombinator.com/item?id=21332957
[3] https://news.ycombinator.com/item?id=20007108
[4] https://www.zotero.org/support/screencast_tutorials/annotation
[5] https://news.ycombinator.com/item?id=18977461