flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
That dependency is now gone, thank you for the review
flatroze's comments
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Everything has its limits
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Could you please open issue on github providing the output that you get in the terminal?
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
It sounds like a great add-on, I have to check it out to see what it does to remote assets and how it works with asynchronously loaded assets.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
I'd say since monolith produces a plaintext document it lets you edit things easier if needed.
JS can be removed from the final document using the -j flag. HTML Files can also be grepped for content, unlike PDFs.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
The compile time is rather long as well, I'm looking into ways of reducing the amount of dependencies.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Thank you for the kind words.
It will evolve into a reliable tool in a couple weeks and it should eventually work for embedding everything, including things like web fonts and @url()'s within CSS. If anything doesn't work, please open an issue, I have plenty of time to work on it.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Which license would you recommend to release this software under to reach the broad adoption yet permissive terms, if not the Unlicense?
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
It is done, option -i in the latest version (2.0.3) now replaces all src="..." attributes with src="<data URL for a transparent PNG pixel>" within IMG tags.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
CSS imports are covered by converting .css files into data URLs, later I will parse those and embed resources found within stylesheets as well.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
It's a valid question. It seems to me users tend to trust things which have certain level of popularity and reputation associated with them.
I personally prefer to hope for the worst. This way when nothing happens I feel extra lucky, and if bad things do happen, I feel proud of being ready for it.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Thank you for the heads up, I'll test it and enhance to preserve styles better.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Thanks, I'll look into making it work with pipes or some other way to interact with headless browsers.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Thank you for reminding me, I need to set action="" to be an absolute path when the page is saved.
upd: Done, now forms get their action="/submit" converted into action="https://website.com/submit" when the page is saved.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Well, at least it's not called iSuck.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Thanks! Pictures should work, I'll check more tags first thing tomorrow when I start working on improving it.
I use youtube-dl for youtube and other popular web services myself. Embedding a video source as a data URL could in theory work, but it'd be quite a long base64 line. Also, editing .html files with tens or hundreds of megabytes of base64 in them would perhaps be less than convenient.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
That's it in the nutshell!
It seems to work for basic pages quite well, I think that lazy load will work for most pages as long as the JavaScript is embedded (no -j flag provided) and the Internet connection is on. It saves what's there when the page is loaded, the rest is a gamble since every website implements infinite scroll differently.
Authentication is another tricky part -- it's different for every browser. I will try to convert it into a web extension of sorts, so that pages could be saved directly from the browser while the user is authenticated.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Uh, I'm sorry. Please feel free to open PRs to that repo if you have anything to add or improve.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
It for sure would help with those SPA websites that get their DOM fully generated by JS. A web extension that saves the current DOM tree as HTML would perhaps do a better job, especially when it comes to resources which require some web-based authentication.
flatroze | 6 years ago | on: Show HN: CLI tool for saving web pages as a single file
Ah, I remember using something like that. I thought that tool was saving it into one .html file, but data URLs didn't exist back then, so creating directories alongside with HTML files was the only option to "replicate" a web resource, now I understand exactly what you were talking about. I'll do some more digging around and implement that in the nearest future. I may need to make all the requests async first to make sure that saving one resource with decent depth won't take too long.
page 1