Little T in the Cog linux

Google App Engine

2018-01-16

After a year and a half of running this site on my Raspberry Pi clueter I've now moved it to Google App Engine. The little cluster of Pi's performed admirably for a while now, but as I'm no longer getting free hosting where it is racked (due to me changing jobs) I can't justify the cost of 1U of rack space here in Auckland so have moved my sites. Most of my stuff is running on Digital Ocean who are an excellent cloud VM provider, but I thought I should finally upskill and get with the program by going serverless. As this website is a simple collection of static HTML content it was an ideal candidate to try this out with.

I've been really happy so far with how it's all worked - it was super easy to set up a new Google App Engine project and get code deployed there. The hardest part was working out how to serve static HTML as it's really geared more towards Node.js/PHP/Python apps and frameworks than static content. Some things still aren't working, for example a 404 on this site used to load my custom 404 page but as I can't set up redirects like I can with a web server I haven't managed to get that working, and so get served a generic 404 error. Also I've had issues running some PHP scripts that expose information about the environment it's running on, but this is probably more by design than an actual error.

The next thing to try will be something a bit more substantial that is backed by a database. I'd really like to move some forums I host to GAE but the product I've been using for years (SMF) doesn't seem like it will be easy to move as it assumes a few things, like directory layouts etc which don't easily translate to serverless. Perhaps it's time to find another forum to host :-)


Installing Ghost on Raspbian

2016-08-08

I recently had a need to install Ghost for a project I'm starting. Ghost is a Node.js app that is a very popular blogging platform. It needs Node.js installed on the server it is running on and can also use a MySQL database backend for storing content. It has excellent documentation but I had to do a few things differently to get it up and running on Raspbian/ARM so I'll document it all here.

The first thing to do is install Node.js. You can use the version shipping with Raspbian as it is supported (currently version 0.10.x, installable via apt-get), but the Ghost folks recommend the LTS version 4 which unfortunately doesn't appear to be in any repos I can find for ARM. Luckily you can download the binaries directly from the Node.js project. You want to install the latest version of 4.x, in my case at the time of writing 4.4.7. the release you want is armv7l for Raspberry Pi 2/3, or if you're still running an older version 1 Pi you'll want to download armv6l. Once you've downloaded it unzip it and copy the binaries and libraries into place on the system somewhere in your path (in my case I copied them to /usr/local/).

curl -O http://nodejs.org/dist/latest-v4.x/node-v4.4.7-linux-armv7l.tar.gz
tar -zxvf node-v4.4.7-linux-armv7l.tar.gz
sudo rsync -av node-v4.4.7-linux-armv7l/bin/ /usr/local/bin/
sudo rsync -av node-v4.4.7-linux-armv7l/include/ /usr/local/include/
sudo rsync -av node-v4.4.7-linux-armv7l/lib/ /usr/local/lib/
sudo rsync -av node-v4.4.7-linux-armv7l/share/ /usr/local/share/

now you've got everything you need to run Ghost (and other Node.js apps) copied into the correct place.

The next step is to install Ghost. For this we create a new user for the app to run as ("ghost"), download and unzip the app, copy it into /var/www/ and run the setup.

sudo useradd -r ghost -U
mkdir ghost && cd ghost
curl -O -L https://ghost.org/zip/ghost-latest.zip
unzip ghost-latest.zip
cd ../ && mv ghost /var/www/
cd /var/www/ghost
sudo chown -R ghost: /var/www/ghost
sudo -u ghost npm install --production
At this point ghost is set up ready to run, though the config file needs a tweak or two. Follow the guide to configure it how you like it (in my case I use MySQL for my blog so all I had to do was add that section, give the blog a URL and add my email address). Once you're happy start the blog by hand to test it:
npm start --production
At this point it should be running on port 2368 on localhost (assuming you left it at the default), so to go further you'll need to configure a reverse proxy to connect to the app. I'm running the excellent Hiawatha webserver so I created a virtualhost like this:
VirtualHost {
	Hostname = myblog.co.nz
	AccessLogfile = /var/log/hiawatha/myblog-access.log
	ErrorLogfile = /var/log/hiawatha/myblog-error.log
	WebsiteRoot = /var/www/ghost
	ReverseProxy .* http://127.0.0.1:2368/
}
If you're running apache then make a vhost somethihng like this:
<VirtualHost *:80>
    ServerName myblog.co.nz
    ServerAlias www.myblog.co.nz
    ProxyPass / http://127.0.0.1:2368/ 
    ProxyPassReverse / http:/127.0.0.1:2368/     
</VirtualHost>
And nginx might look something like:
location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header HOST $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://127.0.0.1:2368;
        proxy_redirect off;
    }
Whatever method you use, at this point you should be able to browse to your blog. If all is well the final step is to create a systemd startup script.
cat <<EOF > /etc/systemd/system/ghost.service
[Unit]
Description=Ghost blog
After=network.target

[Service]
Type=simple
PIDFile=/run/ghost.pid
WorkingDirectory=/var/www/ghost
User=ghost
Group=ghost
ExecStart=/usr/local/bin/npm start --production
ExecStop=/usr/local/bin/npm stop --production
StandardOutput=null
StandardError=null
Restart=always
SyslogIdentifier=Blog Ghost

[Install]
WantedBy=multi-user.target
EOF

You should now be able to enable the ghost server ("sudo systemctl enable ghost.service") and then start/stop etc the blog ("sudo systemctl start ghost.service").


Now Running on Raspberry Pi

2016-08-03

Well, I've finally finished Project Bakery (at least for now - I may tweak it as time goes by) and it is racked up and in production! I'll write up a proper post with photos etc but in essence it is an old Cisco switch that has been gutted and filled with 4x Raspberry Pi 2's all running Raspbian Linux. I've split the functions between the various hosts - I've got one running as a web server using Hiawatha and php-fpm, one running as a database host running MariaDB, one running as an NFS server to provide bulk storage to the cluster (it has a 4TB external disk attached which is then exported to the other hosts) and the final Pi acts as a jump host/build server/SaltStack master.

The whole set up is definitely slower than the Xserve that Owncloud and the various websites were running on, but that's not really the point - it is quick enough for what I want to host and also was fun to design and build. Currently the biggest bottleneck to performance (as far as I can tell) is the DB server works pretty hard when doing things like file scans or Owncloud upgrades, though in general use (browsing files in Owncloud, posting to forums that are hosted there etc) it seems fine. I'll keep an eye on it and upgrade it to a Raspberry Pi 3 if needed (it has taken me so long to actually put the bakery together that the then-current Pi 2's I bought for it have been superseded by the 64bit Pi 3).

toki.co.nz is now ARM powered!

ARM Powered

Upgrading Ubuntu 14.04 to 16.04

2016-04-25

I've just upgraded my first Ubuntu 14.04 box to 16.04. I could have waited around 3 months for Canonical to push it into the update channel and then simply run "do-release-upgrade" but it is easy enough to do manually so thought I'd give it a whirl.

If you want to try this yourself you need to change all instances of "trusty" to "xenial" in /etc/apt/sources.list, eg:

deb http://us.archive.ubuntu.com/ubuntu/ xenial main restricted 
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial main restricted 

Then run "apt-get update && apt-get upgrade". Once it has completed installing the new packages, reboot the box and log back in, then run "apt-get distupgrade". Once those packages are installed (including the new kernel) reboot again. You're now running 16.04!

The machine I picked to do it is a low traffic web server with nothing too flash running on it - no MySQL, no fancy frameworks etc, so I wasn't expecting any issues and for the most part didn't get any. In fact the whole thing went surprisingly smoothly, the only gotcha I encountered was the systemd predictable network interface names which meant that eth0 was now ens3 so the interfaces file no longer worked on boot. You can fix this by changing the line "iface eth0 inet static" to "iface ens3 inet static" or editing the udev rules to name the interface eth0 again. Edit /etc/udev/rules.d/70-persistent-network.rules and change the

KERNEL=="eth*", NAME="eth0"
to
KERNEL=="ens*", NAME="eth0"

and reboot. Once up your interface will be named back to eth0.

3 Days into Chromebook Ownership

2016-03-10

Just a quick report - I'm 3 days into ownership and I simply can't believe how good this little machine is! I've slowly been picking through various apps to install and have settled on a few - Calmly Writer is a beautiful clean minimalist writing app that supports Markdown, Secure Shell is a great SSH client, Polarr is a great offline photo editing app and Evernote has a solid web option. I've even found a WebDAV plugin for the file manager that works perfectly with my Owncloud install so now I have all my documents etc from my other machines accessible without having to move all the files over to Google Drive.

So yes I'm pretty much sold. There are still things that I can't do on this device, for example yesterday I needed to create a bootable SD card image for a Raspbery Pi so had to use my MacBook Pro, and every now and then I need to create USB installers for physical servers, but for the bulk of what I do a Chromebook is a valid device. In fact I'm starting to see the appeal of a Chromebook Pixel and that's something I wouldn't have said a week ago...


New Chromebook

2016-03-07

So I've gone and bought myself a Chromebook. It seems crazy to be sitting here typing this on a small screened, plastic device when I've got a full featured 15" MacBook Pro sitting in the other room but here I am doing just that. The model I ended up with is the HP Chromebook 11 (exact model is 11-2203TU), chosen mostly because it was cheap and not too badly spec'd (it comes with an HDMI port which most cheap Chromebooks seem to leave out and has a slightly faster Celeron rather than the original Exynos ARM chip that powered the early HP Chromebook 11). If I'm honest I'd prefer the original Chromebook 11 package - sleek white with no logos and the four Google colours across the lid but that laptop is considered very slow these days having come out back in 2013 so I'll forsake the more stylish casing for a much faster (and more future proof!) machine.

I must admit I've become slightly obsessed with the gorgeous Chromebook Pixel but figured I'd better try something a wee bit cheaper before diving into a machine that costs roughly $1,500 here in NZ.

- Chrome OS

I'd already done a bunch of research with regards to using Chrome OS so I know that I can do most of what I always have from the stripped out OS - browse the web (obviously!), listen to Spotify, SSH access to servers I manage and some other random apps such as access notes in Evernote (I'm a reasonably heavy user of the service), write documents in Markdown and a few other odds and sods. Some of the Chromebook apps are surprisingly "native" looking, and the whole OS doesn't feel like you're living inside a browser, even if the apps are just web views.

Everything else was already covered - I'm already invested in the Google ecosystem (I mostly use hangouts to chat with people, my email is hosted with GMail, I already use Chrome on my Mac) so there was nothing to set up there.

- HP Chromebook 11 Hardware

The Chromebook 11 is a very cheap laptop so I didn't expect anything too amazing from it. The hardware is nice enough with a compact layout and a good array of ports. The screen is OK - crisp enough on the 11" size but a bit washed out. The battery life is amazing - I've only had the machine a few days but it lasts a whole day with my usage and the WiFi strength is outstanding. The biggest downside to this little workhorse is the lack of RAM - you can really feel it start to bog down when you have a few tabs open and go to a heavy website. I've even had Spotify stutter a few times and drop out when opening some of the heavier apps like GMail when I have several other "apps" already open. More RAM would certainly be welcome.

Overall I'm happy with the device - if the last few days have been any indication then I can probably live with Chrome OS full time. We'll give it a few months and see!


Introducing Project Bakery

2016-01-27

What do you find (among other things) at a bakery? Pies of course! I decided it would be a fun project to put a bunch of Raspberry Pi computers into a small enclosure and put it in a rack as a small compute cluster. This was nicknamed Project Bakery.

I didn’t really have a specific use in mind for this, it was more a fun project but it turns out the tiny ARM based computer makes a capable web server (up to a point - it does only have 1GB RAM after all…) so I figure I can run a website on one, an NFS server on another (with a portable hard drive attached) for storage, a separate database server on another (to give both the web and database server the full 1GB RAM) and finally one last Pi as a general dogs body server (SSH access to the platform, build host, etc).

I was originally going to use a small SuperMicro server chassis for the enclosure, replacing the motherboard with the four Raspberry Pi hosts and re-using the disk caddies along the front, but soon realised that all I would end up with then is a small server that was actually slower than the one it replaced. It also has no style points :-)

I’ve ended up going with an old Cisco 2950 chassis that we had lying around at work (which we could no longer use due to a faulty power supply) as it is a nice compact unit which would almost be invisible if mounted near the top of a rack. I took the switch and gutted the insides, keeping only the fan for cooling. I acquired a sheet of perspex/acrylic to use as a base to mount everything on to neatly and set about looking at what I need for the build.

  1. 4x Raspberry Pi
  2. 1x mini switch (with at least 5 ports, one per Pi and one for the external uplink)
  3. 1x USB hub to distribute power to the Pis
  4. 1x power supply (5volt/5amp)
  5. Miscellaneous cables to cobble it all together (micro USB for power, ethernet for networking etc)

The power supply is something I will probably need to upgrade in the future as I haven’t really left myself much overhead in terms of current draw. I originally thought that 5 amps would be ample as testing has shown that each Pi draws about 240mA when idle and plugged into ethernet and about 500mA when going full chat (compiling etc). So my reckoning was 4x Pi’s running all cores hard would draw about 2A, the original power supply for the switch was only 200mA, a typical external hard drive can burst to 1800mA when spinning up and the fan uses about 400mA when going full speed, giving a total of ~4.4 amps at full draw (it would usually sit much lower than this as the drive would only need that sort of draw when spinning up and it would be very unusual to have all four Pi’s thrashing their CPUs at the same time). Having ordered a power supply small enough to fit in the enclosure that is rated at 5A I then went and read the Raspberry Pi foundation recommended power supply FAQ and they say to be safe each Pi needs about 2A! I’ll set it all up on the 5A power supply and see how I get on, but I may need to revisit this as too little power leads to all sorts of issues with random reboots etc (it’s better to go waaaaay more than you need rather than just enough!).

Anyway, I've now just got to wait for the last bits 'n' pieces to turn up and then I can assemble the whole lot (and take photos while I do it!). I'll hopefull have something to show over the next month or so :-)

Wheezy released

2013-05-06

The latest version of Debian, Wheezy, has just been released. Get it while it's hot!


High-End Chromebook?

2013-02-07

Google's Chromeboook is hardly new - they have been talking up the low end notebook concept for a few years now. The previous efforts by manufacturers were always a bit expensive to really take off (who would pay ~$450US for a notebook that was basically a web browser?) but recent advances in Chrome OS itself which make it more like a "normal" OS coupled with the new Samsung ARM powered Chromebook at only $250US now make it a serious contender for consumer dollars. I've read a bunch of reviews for the new ARM powered Sammy and they have all been fairly positive (being ARM powered there is no fan in the unit meaning it is completely silent and never really gets too hot), to the point that I actually want to get one for myself. I've looked at the apps available and thought about how I use my current ancient PowerBook and think that I could actually use this device for some serious work (we currently use the Google suite of apps at work anyway, the hardest thing to get around will be the lack of a proper shell and the Chrome apps/extensions don't really offer anything quite as good as a traditional bash terminal).

Now that they have hit the price/performance curve and are apparently selling really well (the ARM powered model is forever out of stock and apparently is being used in a few thousand schools in the US) we see this video leak out. I'm not sure if it's a fake or a real product, but if it is real it looks pretty high end (complete with a 2560 x 1700 resolution touch screen) so I doubt it will fall into the cheap 'n' cheerful category like the current ARM powered model does. It'll be interesting if the Chromebook can scale up to compete with "proper" notebooks with a hi-res display and quality construction, but still be in essence a web browser with not much off line ability (though the Google doc suit now supports off line document editing). One interesting prospect is if it is priced well geeks like me may well buy one anyway just to install Ubuntu onto (the same way the current ARM powered device makes an excellent Ubuntu machine) as it could be a cheap hi-res solution.

I hope this does turn out to be real and not just a rumour - and even more importantly than that I hope that it will be for sale here in NZ unlike all the Chromebooks to date...


Will we ever see Android apps on the N9?

2013-01-14

Even before the Nokia N9 was launched back in September 2011 there was talk about getting Android apps to run on it. This isn't actually as crazy as it sounds - both the N9 and Android use a Linux kernel and share a lot of the same userland, only starting to differ once you go up the stack a bit (Android uses the derived-from-Java Dalvik VM whereas the N9 runs MeeGo/QT).

With all the buzz around the new phone being dampened by the fact that there would probably never be another one (due to Nokia jumping into bed with Microsoft) people wondered if there would ever be many apps for the device - queue a bunch of companies saying that they would get the Dalvik VM (and associated libraries) up and running on it so that potentially the whole Android catalogue would be available for it. Some months later several companies boasted that they had it working, but they had no interest selling it to end users, preferring to sell their solutions directly to the OEMs (and it was extremely unlikely that Nokia would be happy putting that on the N9 seeing as they were winding it down at that stage). There was talk of an open source solution but it never really got very far and it looked like it was to remain a pipe dream.

Then in June 2012 OpenMobile put up an interesting blog post showing Android apps running on a MeeGo tablet which generated a lot of feedback with people asking about the N9, so much so that just under 2 weeks later OpenMobile posted:

"Thank you all for your comments and kind words! Our engineering team at OpenMobile is diligently working on multiple platforms, including ACL for the Nokia N9 MeeGo Smartphone. Thank you for your interest in ACL™. Keep checking back to the blog for further updates and information as we are able to release it!"

Well, that was back in June 2012, half a year ago, and though they have shown many demos of their ACL ("Application Compatibility Layer") technology running on different devices there has been no further mention of the N9, and as time goes by it looks increasingly unlikely that they'll ever release anything for it (though apparently Jolla's "Sailfish" OS will use the technology and it is also based on MeeGo so there is still some hope that the engineering effort can somehow be used on the N9 too) and the Android on N9 port I talked about when i first got my N9 is still missing some important features (such as the ability to make a phone call and use the camera) so it looks like running Android apps on the N9 is to remain a pipe dream for a bit longer (forever?).

Does this mean I'm unhappy I bought the N9? Hell no! I'm still really happy with my decision to buy the phone, even though I knew it was end-of-life when I got it. I still haven't seen a phone that I'd rather be using, and until the manufacturers (other than Apple of course!) decide to release a flagship phone at around the 4" size then I'm unlikely to switch any time soon as I'm not a fan of how large the phones are getting now.