Ask HN: How can a intermediate-beginner learn Unix/Linux and programming?

[learnTemp229462]

For a long time, I’ve been in an awkward position with my knowledge of computers. I know basic JavaScript (syntax and booleans and nothing more). I’ve learned the bare basics of Linux from setting up a Pi-hole. I understand the concept of secure hashes. I even know some regex.

The problem is, I know so little that I can’t actually do anything with this knowledge. I suppose I’m looking for a tutorial that will teach me to be comfortable with the command line and a Unix environment, while also teaching me to code a language. Where should I start?

[mooneater]

I disagree with lots of advice here.

My strategy to solidify my linux skills was simple: commit to linux desktop. We spend most of our time on our desktop, may as well let that time earn you skill points. I picked ubuntu and use the same on servers. You naturally learn a bit every day through daily use. This really adds up over time without extra effort of using a side system. Getting good at windows or osx is a waste of your time unless you are specifically into those platforms.

As for a language, pick one that is (1) easy to start with (2) covers advanced use cases, (3) sound in its design, (4) general purpose, (5) concise. That rules out C (due to 1), JS (due to 3 and 4), php (2,3,4) and most things really.

For me that left me with ubuntu and python, I never looked back. (Yes I know many other things but that is my home base.)

[asveikau]

It is trendy to hate C now, but if you want to understand a Unix-like OS I think it is mandatory and they go hand in hand. Otherwise, you will have no idea what kind of interface your higher level language is abstracting.

It's ok to not like it, but don't let that dislike be due to lack of understanding.

[meritt]

JS and php are perfectly suitable general purpose programming languages these days. And given they're both turing complete, they implicitly cover advanced use cases too.

So that leave us with "sound in its design" and I'd argue the absolute unmitigated disaster of python 2 vs 3, the numerous competing competing package managers, and the abundance of really bad code examples due to the influx of the mathematics world (just look at R), I fully disagree that python is a good choice.

But then again, that's just my opinion, but setting people off on an unnecessarily biased path with the assumption that there's only one way to do things definitely sounds like a pythonic behavioral trait.

[roenxi]

Commit to using, breaking and fixing a Linux desktop to learn quickly. I learned a lot running Debian Unstable back back when men were men and unstable was really unstable; so the 2010s.

If minor things are constantly going wrong and getting fixed then the only way that can happen is the user building up a good mental model of how the system works.

An update that breaks X11 will teach you a world of information about terminals. "How do I use a messaging client in a terminal?", "What text editors work in a terminal?", "How do I browse the web from a terminal?", "What games are there for terminals", "How do I watch a movie in the terminal" (ascii).

Get forced to live like that for a week and you also learn how to fix X11 (eg, learning about the difference between 2D and 3D graphics drivers, OSS vs non-OSS drivers, kernel vs userspace). Valuable life skills. Godspeed to the Wayland devs.

[dbcurtis]

Some people are harshing on you for recommending Python because C is the unavoidable end-game in Unix. But I agree with the advice that Python is a great place to start.

Learn to write good Python. After that, learning to write good C will be much faster and easier and far less frustrating. Ultimately, on Unix you end up at C for at least some things (Python is implemented it C, after all). But these days, you can kick the can pretty darn far down the road before needing C.

[empath75]

Counterpoint: I’ve been a Linux sysadmin and developer for close to 10 years at this point and I’ve never used Linux as a desktop. I’ve only used MacBooks.

The skills you need to run Linux on the laptop aren’t particularly close to what you’d need to run Linux as a server, IMO, particularly Linux as a VM.

[fao_]

> , (5) concise. That rules out C (due to 1)

That's interesting, most people I've started on C have found it easier to start out in that than say, Java, which has been on a hell of a lot of introductory courses for a decade now

[jameshe]

Based on personal experience, I can totally echo this comment. I am a business undergrad (senior year), but I am a JavaScript developer for a local start-up. On my Windows laptop, my workflow included applications from MS Office, as well as PowerShell and Vim. At the beginning of this semester, my laptop broke (screen) and I couldn't afford a repair, so I dug up an old ThinkPad T430 that I installed Ubuntu on a couple years back. It was an old machine (2012?) that I bought used, after some searching done on my phone, I upgraded the OS and got to work.

Based on my experience with the switch, I've learned miles more about Linux and how computers work than I would've before. I might even say I like the experience more than I did before. On top of that, as the parent comment says, I've learned more about Linux, shell script, the directory tree, regular expressions, etc., than I think I would've otherwise by self-teaching. It might (read: "definitely will") be uncomfortable at first, but the Linux community is big and helpful and there's plenty of great open-source software that can replace your old workflows.

As far as picking a language goes, I started with Python for the reasons above, but really, starting anywhere is better than being stuck deciding. Common starting points are Python, Java, JavaScript, C#, but really, just pick any language with a large community to help you out. When you think you know enough, consider ideas for a personal project, which is where you'll likely learn the most. After that, keep learning and if you think you want to learn something else, you'll realize how much easier it will be with one language under your belt already.

[Insanity]

100% agree with the 'just use it' part. I've been using Linux machines since the mid 2000's and the knowledge compounds over many years.

For programming language, I'd say Python is decent for a beginner. I've used Python as a language to teach some people programming, and the fact that they can do a wide variety of things (even games with Pygame) helps them stay engaged.

I would always recommend exploring other options as well though, to see what else is out there.

[zelly]

There's no such thing as Linux programming without C. You will have to deal with it eventually regardless of how many wrappers Python etc. puts on top of it.

[RMPR]

I second this, yes you will make mistakes, yes you will break things but that's the way of learning (btw always have a live ISO of your distribution of choice in hand), additionally there's also linuxjourney.com which was an invaluable resource for me (I did the complete journey two or three times).

[29athrowaway]

You can use a Linux desktop today without being exposed to the important stuff.

What is the lifecycle of a process and their states? What is a signal? What is a file descriptor, a socket? etc...

Can you say you know about Linux without being able to answer these questions?

[tmoot]

That's how I learned; on a good ol'fashion gentoo box.

learned so much about os stuff and kernel drivers, etc...

It was like diving in the deep end of a pool though.

[topherPedersen]

That's how I learned as well. I started using Ubuntu Linux on my laptop and was forced to use the command line to install programs.

[unixhero]

Yes. Run a Linux desktop and only stay in it.

[darkteflon]

Completely agree with everything you said - great advice.

[wayneftw]

I agree with this recommendation, but Ubuntu is a PITA for a desktop IMO. Manjaro is much more stable, easier to setup and maintain and the default UI (XFCE) is simpler and very similar to Windows.

Ubuntu is good for a server. Using 2 different distros would be good to learn how not all Unixes/Linuxes are the same and how they might differ.

I would also add: Whatever programming language you learn, you're probably going to want to learn some auxiliary languages as well. Don't get stuck thinking you can do everything with one language. Learn JavaScript, SQL, HTML and CSS as well.

[invisiblerobot]

I agree you need to use it daily but disagree that an Ubuntu desktop provides any real insight.

Here's some BASIC questions an ubuntu user might never encounter:

Which processes are currently hogging the CPU?

Which ports are currently open?

How do you setup a remote git repo and/or add a user for limited ssh?

How do you increase an existing swap part without locking the CPU during write at peak loads?

How do you exclude/include packages from being autoupdated on apt update?

How do you ensure opaque crontabs aren't silently failing?

My advice: Setup a Linux server on aws or whatever for a paying customer with medium load. Only the stress of respondong to actual downtime will teach you anything useful.

[b215826]

The Missing Semester of Your CS Education [1] is a set of tutorials from MIT that gets you to speed with using the CLI, using a text editor (Vim), version controlling (using Git), basic debugging, etc. Some tutorials aren't perfect (e.g., an earlier version had things like `for f in $(ls)` [3]), but at least they link to good tools and resources. There's also Software Carpentry [2], which has similar goals. You might also want to check out The Bash Guide [4,5], which is hands down the best Bash tutorial written.

[1]: https://missing.csail.mit.edu/

[2]: https://software-carpentry.org/lessons/index.html

[3]: https://mywiki.wooledge.org/ParsingLs

[4]: https://mywiki.wooledge.org/BashGuide

[5]: https://guide.bash.academy/

[Waterluvian]

I wonder if introducing newbies to makefiles and vim is super empowering, or ends up gatekeeping by making it look "as complicated as you feared it would be."

I feel like before any of that:

1. Do practical stuff with Python

2. Learn the semantics behind the Linux directory tree

3. CLI, piping stuff together, a little bit of bash.

[stank345]

My advice is to start working on a project you care about and then learn everything required to make it happen. If you already know JavaScript, maybe build and deploy a static website. Once you've done that then perhaps make it interact with an API.

[jordanpg]

This. Having something to actually accomplish is much more important than finding the ultimate book or online course. There are lots of those.

Pick one and move on to defining a realistic project that seems interesting and achievable.

Most important of all, finish it, and write what you did down somewhere. It took me 20 years to realize how important this is.

[jimnotgym]

If you are on Windows I recommend setting up Windows Subsystem for Linux. With VScode. You now have a Linux server with remote code editing. I note you have an Raspberry pi, what follows will work with the pi as well with whatever remote machine. Complete these tasks

1) install the language of your choice and start a simple web site using the languages webserver. View the website in your browser on Windows. A Python Flask 'hello world' or a Node one?

2) Set up an SSH server and remote in from the Windows machine. Learn how to set it up to use keys only instead of passwords. Find out why that is a good idea

3) learn Git. Github has some nice tutorials.

4) set up NGinx to run your website above.

5) set up Nginx for https

6) set up LetsEncrypt.

7) set up a free AWS account, start a server and put your website on it using SSH and git.

8) look through the ssh log files to see login attempts, and the Nginx logs. Imagine there were millions of rows, how would you find a certain one, or the last few? (head, tail, grep, cat).

That should get you somewhere.!

[bigmit37]

Thanks this is helpful to get started.

[pbh101]

Some resources I've found helpful to learn some command-line:

- https://cmdchallenge.com/. This set of increasing-in-difficulty challenges should help with structure in terms of learning a bit more.

- https://explainshell.com/ can be helpful as well.

- Don't forget the man pages! `man <command>` can be quite helpful! For a different approach on teaching some, bropages.org could be helpful.

- I haven't used it personally, but there is a lot of info at http://www.opsschool.org/

As for learning to program, I suggest a very good starter language is Python, and its built-in, first-party language reference/tutorial I found helpful when I learned it: https://docs.python.org/3.8/tutorial/index.html

As for "what" to program after that, I don't have any great resources for what constitutes some good initial programs. Off the top of my head, here are some ideas about places to start which are doable with only inputting and outputting text:

- FizzBuzz

- Play Rock Paper Scissors with the computer

- Play Tic Tac Toe with the Computer

- Sieve of Eratosthenes

- Conway's Game of Life

- Given a piece of text, count how many occurrences of each word there are and display the top ten.

[deevnullx]

My approach to teaching Linux and programming leans heavily on using existing resources and ensuring students get a wide base of knowledge before beginning the "exploratory" stage of learning.

Start with installing a distro of your choice in a virtualization tool, I always recommend Ubuntu on VMWarePlayer because it was what I started with. Distro/Virtualization Software doesn't matter, what matters is that you use it.

Then I have students work through Linux Journey [1] one module at a time, while being available to answer questions as they go.

Once a student has finished the LJ modules I have them play the Bandit Over the Wire [2] game up to challenge 15.

After that I usually teach Hardware and Operating Systems, but if you are just looking for general ability to do things, skip to the CyberAces Linux modules. [3]

Once you have finished those, do the CyberAces Bash Scripting module and the CodeAcademy Python [4] course and you'll have enough scripting ability to be dangerous.

From there, just keep using Linux as your daily driver and you will continue to improve, especially if you take on programming projects.

All of this can be done by yourself, but I teach a free online course on Computing Fundamentals and Security if you are interested in it being more of a guided experience. [5]

[1] https://linuxjourney.com/ [2] http://overthewire.org/wargames/bandit/ [3] https://tutorials.cyberaces.org/tutorials.html [4] https://www.codecademy.com/learn/learn-python [5] https://www.hoppersroppers.org/course.html

[DyslexicAtheist]

a really good book is Richard Stevens Advanced Programming in the UNIX environment[1]. (sounds daunting but it's not too bad when you combine it with another introductory text on C). If you stick with C you'll eventually know UNIX/Linux a lot deeper than anyone. It takes time though so go easy on yourself.

Also check github for a bunch of repos that contain biolerplate code that is used in most deamons illustrating signal handling, forking, etc.[2]

Also I suggest taking some open source projects from Daniel Bernstein (DJB) as examples on how to write secure code. qmail, djbdns, daemontools ... there are lots of great ideas there[3]

Write a couple of simple programs that utilize your code and expand from there, learn the plumbing, e.g. write a Makefile, learn autotools/autoconf, how to write tests, how to use gdb, how to create shared libs and link a program LD_LIBRARY_PATH/ldconfig, etc ...

Most important think about something that you like to write and go from there. Back in the late 90ies I studied RFC's (MIME, SMTP, etc) and then implemented things that I could actually use myself. Here is a recent project I did some weeks ago (an extreme edge-case for security maximalists which will never be used by anyone other then myself, ... but I really enjoyed writing this and learned a bunch of new stuff while doing so)[4]

if you need ideas or help with looking at your code, don't hesitate and send me a mail.

[1] https://en.wikipedia.org/wiki/Advanced_Programming_in_the_Un...

[2] https://github.com/jirihnidek/daemon

[3] https://cr.yp.to/djb.html

[4] https://github.com/DyslexicAtheist/nfq/

[westurner]

> Also check github for a bunch of repos that contain biolerplate code that is used in most deamons illustrating signal handling, forking, etc.[2]

docker-systemctl-replacement is a (partial) reimplementation of systemd as one python script that can be run as the init process of a container that's helpful for understanding how systemd handles processes: https://github.com/gdraheim/docker-systemctl-replacement/blo...

systemd is written in C: https://github.com/systemd/systemd

> examples of how to write secure code

awesome-safety-critical > Coding Guidelines https://github.com/stanislaw/awesome-safety-critical/blob/ma...

[osrec]

Download Ubuntu and run it as a virtual machine inside your current machine (use Google for instructions - it's fairly straightforward with a piece of software called virtualbox). That should stop you from running into hardware related compatibility issues.

Get familiar with the interface and the concept of a "terminal".

Then try do some simple stuff. Maybe set up nginx (or Apache) to run a web server that you can access via your host machine.

Thereafter try doing some Linux admin tasks, like creating users, switching users. Maybe even set up SSH and try accessing the virtual machine from your host machine via PuTTY.

After, maybe try spinning up a simple node web app on the virtual machine and access it from a browser on your host machine.

By this time, you'll probably get a sense of what you want to explore next! And remember, Google it's your friend for all of the above!

All the best :)

[learnTemp229462]

I’ll probably use my raspberry pi instead of installing a VM.

[towb]

Jump in on the deep end. Install Arch, a tiling wm of some sort, and make it yours. Soon enough you realise this isn't the deep end anymore. You get a lot for free by just using linux every day and dealing with whatever problems might pop up.

Keep the terminal open, if it can be done in a terminal it may turn out you prefer it when you get used to it. Torrents, IRC, playing music, email, file explorer, whatever makes you stay in there. It makes you comfortable navigating the system and you get to see what a GUI hides from you.

Programming? You're not gonna benefit from diving into any deep ends there, continue with JS if that's what you like. Learn databases and web servers, here your new linux skills comes in use. Try some stuff and see what you like.

[smabie]

I would recommend learning Linux the way I did 15 or so years ago: get rid of other operating systems and use a more 'advanced' distribution (such as Arch or Gentoo). In my case, I fucked up my Windows install on a new laptop when I was 12. At that point, I was stuck and just decided to stick with Linux. It's remarkable how quickly you learn when you absolutely need to in order to use the computer.

My second piece of advice would be to learn C from this book: https://en.wikipedia.org/wiki/The_C_Programming_Language

After finishing it and completing all the exercises, you should be ready to go learn anything else you want to.

[mgerdts]

Better is the way I learned it 25 years ago. Burn up your 386sx CPU while doing a kernel build, then get a loan for a nicer computer. It should have at least 4 MB of ram. Spend about 200x minimum houry wage on a nice monitor, then be very careful with manual modeline settings in your xfree86 config so that you don't destroy the monitor.

Get yourself a bulldog clip to hold that send book open to the page you are repeated referring to. When you graduate to Stephens' UNIX Systems Programming, the bulldog clip won't be needed.

If your monitor doesn't stand tall enough, 4 reams of copy paper are much cheaper than the same thickness of X programming manuals.

[goshx]

For programming, start by having a project. Think about something you’d like to build and focus on one thing at a time. This will guide your research.

For Unix, install linux on your computer and use it daily. You’ll be forced to learn.

[bluemooner]

Here's a couple of tutorials on various tools that might be useful for a beginner: https://missing.csail.mit.edu/

[ryansmccoy]

Raspberry Pi and free trials to O'Reilly Online (which I currently now subscribe and don't need credit card to sign up for). They've been giving 50% off Black Friday Special, so keep eye out for that.

Here are some of my favorite books:

https://learning.oreilly.com/library/view/ubuntu-server-cook... - really good one for learning real uses of linux

https://learning.oreilly.com/library/view/raspberry-pi-cookb...

https://learning.oreilly.com/library/view/raspberry-pi-netwo...

https://learning.oreilly.com/library/view/exploring-raspberr...

https://learning.oreilly.com/library/view/raspberry-pi-hacks...

[ShiroiAkuma]

Its very easy to get good with unix. I usually use the tee rule for this (look up tee on google ;) )

1) Get a good linux, you can start with ubuntu. After getting comfy with ubuntu try another os, debian,And eventually to the hardcore ones like Arch, Artix and others. Try to do your usual thing with commandline. For example, How to copy using terminal,How to open a file with terminal, How to get lots of terminals inside terminal. Stuffs like this (trick is simply go ahead and add "How to do it in terminal" ;) ) Google them up. (www.giyf.com) Try to rice your system. It teaches crucial aspects about the appearance,usability etc.

2) Get a good book. No starch press has some good ones, even O reilly gets you into shape. The trick to good book is simple: Get a book, read it. If you love it finish it. There are open source notes as well. Awesome cli github is similar.

3) Try to create your own tools using 1,2 this will allow you to bend your brain. Stuffs like How can I get hackernews comments in my terminal, How can I create a shell script to remind me that 20mins have passed (pomodoro) and stuffs like that.

I have some starred repositories and Some simple tools of my own (Aeres-u99 @github) to check for some example.

I think with this (depending how extensively you do them) you can be decent

[morty_s]

There are many ways, but here is one:

Read and lightly work through The C Programming Language (K&R). Code along with the book, don’t feel you have to really deep-dive anything.

You don’t have to learn C well at this point. Just learn from it.

Then start reading The Art Of Unix Programming. Do this casually. Read a bit, code a bit. Between K&R and TAOUP, you’ll be inspired by to write at least two small projects.

To supplement K&R and TAOUP, begin reading Unix Network Programming.

If you have access to a Mac or a Linux machine, explore the man pages.

If you’re on a Mac, explore the man pages __and__ the code that lives on your system.

One way to do this is to use something like vscode’s intellisense to “Go to Definition.” E.g., when you’re coding an example from K&R, right click on one of the included headers, and click “Go to definition.” It will take you to the header file. Look through the code. Then you could look at the code from FreeBSD or OpenBSD on GitHub.

Edit: Just wanted to add, by doing the above you’ll get exposure to your system in new ways. Moreover, you’ll find yourself learning Unix out of necessity, for fun, and by accident. You can practice the art of Unix programming in almost any language at this point. C is unix-y, but so is Rust.

[starpilot]

Use Slackware, do these exercises:

- Customize your bash prompt, create aliases for some common commans (ls, df are good)

- Build a LAMP stack, compile everything from source tarballs. Get it working so that you're able to browse your webserver from your phone.

- Update a sound or video driver

- After you're comfortable, recompile the kernel

- Through these, do as much in the command line as you can. Get comfortable with mv rm chmod chown and so on.

Any time you have edit a text file, use vim, nothing else.

[emmelaich]

I strongly recommend Rochkind - Advanced Unix Programming.

Despite the title, it's not that advanced; the "advanced" here refers to system-level programming, not application.

Note, for this you probably need to learn C.

K&R ed2 remains an excellent book for this.

Honorary mention - Kernighan and Pike - Unix Programming Environment. But it's a little dated.

Above all, do. Get a Ubuntu or Centos image, install developer tools (gcc), manpages, and write some code.

[pixel_tek]

Hm, in my case, I followed a very boring/the right way of doing things, For example : try getting the book "the Linux programming interface" it s very handy and explains pretty much everything u de want to know about anything in a Linux operation system, it lists and explains the POSIX interface made for you to use in you C code , with example, which surprisingly, u get to find similar interfaces/system calls/library functions that you can use in python to do the same thing.

For regular expressions, watching a small video which explains how they work is definitely more productive, but if you could find some time to spare , try checking how compilers are made, / what constitutes a language (language theory), and automata, it definitely has some math in it , but it s worth it, it gives you an extra mental image , which helps you get a better understanding of where you stand and what you can do. Since apparently regex is also a language, which helps you describe chunks of an other languages and it also has some limitations to be aware of.

[bigmit37]

I’m primarily a windows user and am programming to create products. Since I don’t work a programmer job, my learning is unstructured. Is bash the main benefit of learning something like Ubuntu?

I have delayed making the switch because I have lot of things to learn (currently relearning math, testing out different deep learning architectures, and learning JavaScript (second language after Python)) and not sure if should prioritize learning it.

I still need to get better with GitHub which is something I clearly see the benefits of. Also need to get better with a text editor , using Visual Studio Code , mainly just for text completion in Python and JavaScript, so need to learn the additional benefits. Also want to test sublime text.

I see recommendations for vim, but I wonder if that will make me too twitchy and will start to type before thinking.

So due to the amount of things I am trying to learn, I have delayed learning Ubuntu.

I know Windows 10 now has built-in bash capabilities but I wonder if there are other benefits I am missing with Linux and worth learning.

[john-radio]

> I’m primarily a windows user and am programming to create products. Since I don’t work a programmer job, my learning is unstructured. Is bash the main benefit of learning something like Ubuntu?

That's one advantage. The big advantage is that most enterprise web sites and webservices run either on a rented Linux server or on a Linux instance that's "serverless" (meaning it can run processes and access databases, but can't save stuff permanently to the filesystem). If you have familiarity with Linux, then that lends itself well to the work of troubleshooting the complications that come with that setup.

[baryphonic]

I learned SunOS (and later Linux and HP-UX) by full immersion with a non-root account on a multiuser server. It was non-graphical, though I was able to tunnel X over SSH across the Internet and actually have an almost usable desktop.

I was learning C programming at the time, so beyond dealing with bizarre things like my program crashing due to "segmentation fault"s, I was trying to understand what was going on. The first time I opened vi, I couldn't figure out how to get out. Once I did, I used `:sav %` followed by `:q` for longer than I care to admit.

If I had to recommend anything to someone just learning, I'd recommend learning these commands:

cd ls touch rm # but be careful! ps head tail nano who whoami man apropos # this is technically man less

These will help you do a lot of what you need to do on Linux about as well as you can do similar things on Windows. Once you've mastered those commands, learn about piping `|`, grouping `{ foo; bar; }` and sub-shells `$(foo)`. Learning about variables `$foo`

Then read up on:

grep sed cut find awk # really just `awk '{ print $2 }'` xargs env export test # [ foo ] source # . foo crontab top # or even better `htop` shell control flow

When you're done with that, learn

vim # `vimtutor` can help emacs

After that, there's tons of other tools you'll pick up as needed:

dd df du fsck and a bunch of other system-related tools

You'll learn a ton as you move along.

Just remember that for the most part, each tool solves approximately one problem, and the complex behavior comes from chaining tools together. Chaining mostly involves `|`. Once you see that at work, you probably won't want to go back to the crutch of point-click-drag.

Best of luck!