Trolls are breaking the Stack Exchange system: part 1

If a question is popular and gets a lot of attention, but is on a slightly complex topic, what will happen? Enough stupid users will look at it, miss-interpret it and vote to close it. It just takes 5 idiots over any length of time. But then to get it re-opened, it’s near impossible. It takes 5 users over a short time. Yes, there’s a time limit, but only for reopening. And of course, once a question’s closed, views plummet because the SE system makes it so. So that question is then dead. A valuable question which brings a challenging concept [to some] … is permanently dead.

And what if you delete and start again? You get penalised. You can even get banned. What happens if you ask it a second time? It gets marked as a duplicate.

This is just one way  in which the SE system is broken. And it is trolls that are breaking it.

Rant over. For now.

 

 

Turn RaspberryPi-UA-NetInst into a NodeJS server

Rather than reduce Raspbian by uninstalling packages and hacking down services run at boot-up to give a minimal system that runs NodeJS, I thought I’d try starting with a minimal system using RaspberryPi-UA-NetInst and just adding what’s needed.

That link gives an excellent tutorial on getting started. But before you install it, there are tweaks to be made before installing it that will get a lot of stuff set up on installing, rather than afterwards. This is a great bonus because if you ever have to reinstall, you don’t have to reconfigure the system each time. Simply format the SD card, place your modified installer files on the card, put it in the RPi and boot up, cross fingers and it’ll all be installed in under 30 minutes. And it can all be done over WiFi without touching a network cable!

So simple! Just follow Method#1 in this previous tutorial of mine, but replace the contents of  on-first-login.sh with the following:

You might want to tweak the installer to create a user besides root, it seems to be a preferred way of doing things! But I’ll leave that to you. From here you can test out your new NodeJS server, check out this NodeJS tutorial.

This can be taken a few steps further still though! What if you want your entire NodeJS Pi server set up un an unattended fashion, right up to downloading your NodeJS source code from GitHub and running the server? And configuring systemd so it starts up automatically every time? No problem! Tutorial coming soon! 😉

 

NodeJS web server bits and pieces

Disadvantages of NodeJs:

Avoid asynchronous he’ll with Node Fibers:

Web framework for NodeJs:

GPIO access with NodeJs:

Customise Raspbian-Ua-NetInst – beyond post-install.txt

Want a bare-bones Raspbian distribution? Start with this one:

  • Raspbian-Ua-NetInst (Smaller installer)
  • RaspberryPi-Ua-NetInst (Firstly, it installs over wifi and gives faster boot time – 10 seconds! but mainly it lets you “Bring your own” files, which we’ll need for method 1 below.)

But what if you want to make a custom installer which comes with more? With either of the above, there are two ways you can do this:

  1. Add package names to installer-configure.txt as per the instructions on those links. Easy!
  2. Add scripts to post-install.txt  to manipulate what’s been installed.

The problem is the former does not give scope for configuration, and the latter is somewhat of a black art. For example, in post-install.txt you cannot simply include commands like this:

Instead you need to do this:

And if you want to manipulate files that will appear on the final installation, God help you. I have had mixed success with this. YMMV!

This brings me to the greatest resource for moving forward:

There are three options it hints at:

  1. Add a script which executes on first login using /root/.bashrc. ,using systemd to log in automatically on first boot to run it.
  2. Use  post-install.txt to install your own custom repository.
  3. Cron: set up a cronjob to run a script on boot-up
  4. Use chef or similar, like this guy did

I will explore them in that order….

Method 1:

This assumes that you’ll be using the Pi’s UART via a USB-serial console, rather than using SSH to log in.

First, get a copy of the raspbian-ua-netinst installer, and follow the instructions, all from here: RaspberryPi-Ua-NetInst.

Everything we need to do will involve adding and changing files they provide that will be going onto your SD card before you put it in your Pi.

They way this method works is first we will configure the installer by making a configuration file in the root of the SD card called installer-config.txt . Put the following in it to make output go to the UART:

Once the installer has done most of the installing, we will tack on a few things for it to do by adding a file at the root of the SD card called post-install.txt . It will run these commands before the installer after it’s done the installing, but before it reboots.

The final feature of the installer that we will use, is it allows us to bundle any files we want onto the SD card, and tell it where it should place them on the final system. We do this by placing them in /config/files/ , and listing them in /config/my-files.txt  as per the instructions.

Putting it all together, we will use these features of the installer to firstly make the system automatically log in once the installer reboots, then to have the system run a script we’ll make once it’s logged in.

We will adapt this tutorial which shows how to make the Pi log in automatically when boot up after installation, using systemd, which we will configure to do so simply by adding a file we’ll call  serial-getty@ttyAMA0.service  into two places within /config/files .

Secondly, when the system is installed and boots and automatically logs-in for the first time, we’ll get the system to run our commands by putting a little something into the .bashrc file in root’s home directory. Here’s how:

In the installer files that you copy on to the SD card, edit /config/my-files.list  and add these lines:

Create this directory structure:

Make two files, both called  serial-getty@ttyAMA0.service , in the following locations:

With the following in each:

Then make a file called on-first-login.sh  in this location:

With the following contents:

Make a file at the root of the installation files called add.bashrc with the following:

Now create a file at the root of the installation files called post-install.txt  and put this in it:

And that’s it! Now you have what you need to easily set up your system post-installation without any attention required – it will do the entire thing without any input, just insert the SD card, power-up and wait 20 minutes! Just put your post-installation commands in on-first-login.sh  .

Lol @ linux

Why is Linux so hard? You have to do a lot of rooting around.

(Sorry… worst joke yet)

Installing Node.JS v4.0 on RPi

As usual, tutorials on Linux become out of date quickly. And as usual, linux tutorials always need fiddling to get working on any setup different to that of the author’s. So here is how I got node v4.0 installed on the RPi version B with Raspbian:

First, from this tutorial:

Then, from this tutorial:

Then edit ~/.bashrc:

Add this line  at the end:

Then edit ~/.profile:

Add this line at the end:

Reboot:

Finally you can get on with actually using it!

But if you want a slimmed-down Raspbian, try my tutorial on getting RaspberryPi-Ua-NetInst working with NodeJs.

This tutorial has a great introduction to using Node.JS. It requires NodeJS v4.0 which you should now have. Plus you now have nvm , the node version manager, which it also requires.

 

My love-hate relationship with Linux

I have been programming for about 20 years. Within that, about a solid year has been fiddling with Linux. Yet I wonder how much I’ve been able to remember. When ever I try to set something up in Linux, I quickly start swearing like a trooper, and end up Googling things like “I f****ing hate Linux”. Or “Why is Linux so f***ing obscure”. It’s reassuring to know I’m not alone.

But why do I keep going back? Because it has its place. While I think it is unfit as a desktop OS for 99.9% of cases, (stats don’t lie. How many people actually use it for that?) The fact is it is beyond widespread on servers, and frequently used for embedded systems and systems like the Raspberry Pi. If you know what you’re doing, (if you *really* know what you’re doing), you can butcher it, configure it and extend it to your heart’s content. Oh, and it’s free. That’s why I love it.

You just need to read through the comments on this blog post to see plenty of reasons Linux is hated. But I’m going to add a couple. I’m fiddling with an RPi at the moment, so I will have plenty of reminders for what to say on this matter.

The truth is that whenever I try something remotely ambitious in Linux, I end up throwing in the towel. We shall see how this goes.

Have you tried understanding entries in /etc/inittab  ? Even once you know how it works, there’s no denying that the syntax is cryptic compared to how modern programming languages might specify the same configurations. It’s hardly as ubiquitous as JSON.

Why is it so hard to get things working on Linux? The problem is threefold:

  1. Those very comfortable with Linux who write tutorials assume that readers know everything but the tip of the problem, so skip on the details.
  2. Those not comfortable will have spent so much time and tried so many commands to try to get it to work that by the time they write the tutorial they’ve lost track of 95% of the things they had to try first.
  3. Because of the wild permutations of possible distros, versions, configurations and hardware, very rarely are two people’s situations the same.
  4. Combine reason 1 and 3: those in the know can’t, without huge effort, write a tutorial or make a script to, for example, make WiFi work. So many possible combinations, software always changing, it wouldn’t work for long.

 

RPi stripped

These are some options for using a stripped-down RPi when you need the best performance by tailoring what you need to the bare minimum.

Why stripped down? If you want it fast. Fast boot, fast processing, low latency. The RPi is in the order of 100 times faster than 8-bit PIC microcontrollers that I’m used to using, yet most Linux distributions adds a lot of bloat for these kinds of use-cases.

In order from high to low level:

Raspbian: It’s well supported; regular updates and plethora of available packages. Also, packages can quickly be removed as per this previous post, and boot time and other run-time resource hogs can be eliminated as per this previous post.

Why not Arch linux? It’s an option. However, I spent a day just trying to get it to boot, with no luck. It is no longer an option in the NOOBS installer, and having no previous experience with it, and little experience with the RPi, I’d lost before I started.

Start with Raspbian-Ua-NetInst and build it up: If the experience with Arch was too ambitious, then this probably is too. While it is dead easy to install, I wouldn’t want to spend months and months learning with no guarantee of success; I’d want to start with something that just works – even if mediocre – and take it from there. It doesn’t even come with a package manager, and a quick Google search shows that the lack of tutorials for building on it might be a deal-breaker.

Why not MiniBian? ‘Cos I just found out about it.

PiLFS: Probably the lowest level besides bare-metal, but still pretty bulky. It’s not all that clear how much space it would take on the SD card; somewhere between 8MB and 1GB as a minimum??

Why not bare-metal? Are you kidding me? I don’t want to write a Web server, or networking capability from scratch. The reason I’ve decided to use a RPi is to give powerful high-level programming capability to really get stuff done. However, a great tutorial is here: http://www.valvers.com/open-software/raspberry-pi/step01-bare-metal-programming-in-cpt1/. It is a massive 5-part tutorial which gets you a long way, relatively speaking; great if you just want a quick way of hammering hardware.

 

 

Raspbian – resources for reducing boot time

This post will be a work-in-progress. How to reduce boot time of your Raspberry Pi running Raspbian?

DMesg

First you need to know how long it actually takes to boot up so you can assess your progress.

It has the added benefit of telling you what is happening in that time. Run that command and it will give an output like this:

The number in square brackets on the left is the time since it started up, and whatever is on the right is what is being loaded at that point. I’m not sure what the very last line is about, but the line before that indicates that boot-up time (including ethernet setup in this case) was 22.1 seconds.

Stop stuff being done at boot up

This topic on the Raspberry Pi Forums gives a few ideas for how to speed boot-up time. There are an awful lot of good responses there. Things I’ve tried / will try:

  1. Install rcconf; the Debian Runlevel Configuration tool
  2. Edit /boot/cmdline.txt , add kernel option quiet so boot up log is not printed
  3. In that same file, choose fastboot option which bypasses filesystem check. Supposedly a bad idea though.

Another link: http://rpitc.blogspot.co.uk/p/docs.html

Random hint: some things are never documented. Except in the source code itself. For example, if you have to edit /boot/cmdline.txt , then the only help you will get is by looking at the source.

There are different ways of changing what services are started up during boot-up. I’m not clear what’s supposed to be used when; the complication arises because the way boot-up services are run has changed over the course of Raspbian’s existance. Earlier versions used /etc/inittab but newer versions use systemd . I think this change is why rcconf and sysv-rc-conf don’t work properly; they seem to lose settings on reboot, which makes them useless.

The most successful way I’ve had of reducing boot times is by using  systemd-analyze blame  to show what is taking longest to load. It outputs something like this:

To find out what the services you’re disabling actually do, run this:

And you’ll get an output like this: (Scroll to the right to see the description).

Then disable the individual services as you desire by doing this: