Setup Raspberry PI 3 with no (ethernet) cable, monitor, keyboard, etc.

A complete guide to setting up a Raspberry Pi 3 (RPi) to connect wirelessly. Everything is setup from your laptop, writing to a micro SD card before plugging the card in to the RPi. I'm doing this on an Ubuntu computer. I imagine it is very similar from a Mac.

  1. Download and unzip the Raspbian Lite operating system (OS). You will now have an img file ready to write somewhere.
  2. Download, install and run Etcher to write the raw operating system to the SD card. This is by far the easiest / safest way to write the OS to the card. (Note I have no liability for mistakes that may occur with this. Follow the instructions on Etcher's website)
  3. Open a command window and enter:

    where xyz is the path to the SD card and main_drive_abc will be some long text. One of these two is not necessary but I don't care which. This writes an empty file to these folders. They enable ssh so the RPi can be connected to using the SSH protocol from your laptop.
  4. Now to use WiFi you'll need to give the RPi your username and password. This command will open an editor on the file that stores this information:

    At the bottom of the file add this:

    Now press Ctrl+X to exit. Press "y" to save it as the correct filename.
  5. Put the SD card in the RPi. Plug in the USB power cable. You are ready to connect from your laptop.
  6. Find the RPi on your network using the command
    sudo arp-scan --localnet
    Hopefully there will be an obvious choice. If not, try the following instruction changing the final number in the IP address below with each choice you see in arp-scan's output
  7. Connect to the RPi with the command (changing 192.168.0.18 for whatever number your RPi is allocated on your network)

    The username you are using is pi. The password when prompted is "raspberry"
  8. It is a huge security whole leaving the password unchanged. Run

    This gives you an interactive menu. Change the password, update the system. You now have an operational Raspberry Pi to hack.

Swimming times

Options for lunchtime lane swimming near Canary Wharf (and NW London in the evening)

Tiller Road Burdett Road St Georges Swiss Cottage Northolt
Mon 12:15 - 13:30 12:15 - 21:30 12:30 - 13:30 06:30 - 21:30
Tues 12:15 - 13:30 12:00 - 21:30 12:30 - 13:30 2 lanes 06:30 - 21:30
Wed 12:00 - 13:30 12:00 - 21:30 12:30 - 13:30 06:30 - 21:30
Thurs 12:15 - 13:30 12:00 - 19:00 None 06:30 - 21:30
Fri 12:15 - 13:30 06:30 - 22:00 12:30 - 13:30 06:30 - 18:00 06:30 - 21:30

Pool hours rather than centre hours

RPi Tube status checker

Like most Londoners I depend on the Underground system to get to the office where I am on a contract. A delay or suspension can cause big problems for the morning commute so I decided to put my Raspberry Pi to use, alerting of problems on the tube network before heading out the door. I used TFL's API: status to check how the lines are running and put the Piglow I had to use – one arm for the Metropolitan line, one for the Jubilee line, the final arm for the other lines combined (I get to work using just the Met and Jubilee lines). A single light on an arm of the Piglow indicates good service, two lights - minor delays and the whole arm lit to indicate more serious problems.

Raspberry Pi showing tube status
Raspberry Pi showing London underground status

defaultdict explained

If we wanted to count occurrences in a list (without collections.Counter) we could write this:

Alternatively, we could use a defaultdict. The constructor of a defaultdict takes one arg – a function – what to do when a new key is added to the dict. So instead, you could write:

But the final addition that makes it briefer but perhaps more confusing on first approach is that defaultdict understands int to mean the same as lambda: 0 and has the same effect. I.e.

How to price a swap

A vanilla interest rate swap is an agreement to swap a fixed rate coupons made on a large notional amount against floating rate coupons on the same notional. The floating rate used may be LIBOR, EURIBOR, EONIA, etc. At the conclusion of the agreement, the notional amount of the trade is not exchanged between the parties.

Our example will be a five year swap where payments are made every 6 months using LIBOR for the floating leg and the fixed rate will be 3%. On the trade date the LIBOR rate will be recorded. This will dictate the floating payment payment to be made at the first payment date in 6 months time. At that time, the corresponding fixed payment will be 0.03 * notional_amount. For the other 9 payments, the fixed leg will continue to pay the same sum: 0.03 * notional_amount. For the floating leg, the rate to be paid will continue to be the LIBOR rate snapped at the previous payment date i.e.

The price of each leg is as follows:

PV = ce- 0.5 R1 + ce- 1.0 R2

Where c (for coupon) is the notional amount

So the value of the fixed leg is a straight forward calculation. The value of the floating leg is the same but uses rates taken from the swap curve which we need to build.

Building a swap curve

When valuing the floating leg on the trade date, we need to take rates from the swap curve.Swap curve
This is built from a variety of products taken from exchanges where the products are of similar credit worthiness.

The rates that make the curve are "n-year zero rates", often called zero rates. A zero rate is the rate an investment (like an IR bond) would pay when all of the interest was paid at maturity with the repayment of a principal amount. So we can see a 5 year investment should return at a rate of about 4.2% from our curve above. Where do these rates to make the curve come from:

Short-end

These are the easy points of the swap curve. In the near future, there are many publicly quoted bonds that are nearing expiry where there are no remaining coupons to be paid. It can therefore be assumed that the rates offered are purely zero-rates. They just need to be converted in to continually compounding interest.

A bond with a principal of $100, maturing in 3 months (0.25 years) has no more coupons to pay and in the market today is priced at $97.50. Therefore it will return $2.5 over the 3 months and so its quarterly compounding interest is

(4 x 2.5 / 97.5 = 0.10256 or 10.256% per annum

Expressed in continually compounding interest, this is

R3m = 4 ln (1 + (0.10256 / 4)) = 0.10127 or  10.127%

Long-end

Further out in the future, bonds will have coupons still to be paid midway through their lifecycle. These can make points on the swap curve but first have to be converted to a zero-rate IR products: adjusted to be a single bullet payment at maturity with the correct equivalent interest rate. This new interest rate is called the "implied zero-coupon rate" or just "zero rate". Bonds are added to the curve one after another from the earlier maturing bonds to the last maturing, going out 30 - 50 years from now. To take an example:

We have a bond expiring in 6 months (0.5 years). It pays 4 coupons annually of $8 ($2 each time) with a $100 principal. Today it is available on the market at $96.00. Therefore the payments on this bond will be:

3 months: $2
6 months: $102

So for the zero rate at 6 months, this equation allows us to infer the 6m continuously compounding rate:

2 e0.10127 x 0.25 + 2 e0.5 R = 96

∴  R6m = 2ln(96 - 2 e0.10127 x 0.25 / 2)

R vs Python: memory use

I've recently been trying to build some machine learning skills by working on prediction challenges from Kaggle. Getting some R skills was also a goal. I was trying to work on the San Francisco Crime project and planned to start by growing a Random Forest for the full training set dropping some of the predictors (n=878050, p=5).

After debugging the code with a small sample of the data, I ran with the full dataset and my laptop ran out of memory. I switched to the Google Cloud Compute instance I'm trialling: 2 CPUs, 13GB memory + 7GB of swap. Still not enough memory to build a single-tree Random Forest in R. Next I took the training data and tried to build Random Forests for one of the 10 Districts. Again I couldn't build the forest due to insufficient memory on the big Google instance I had.

Thinking this was getting ridiculous, I ported the code to Python to use Scikit-Learn. I ran the same single-tree random forest; one forest for each district in memory at the same time and the model built fine on my low-spec laptop sharing memory with Chromium, OpenOffice, etc. I intentionally avoided optimising the Python code so it had no head-start on the R code. These are the two versions of the random forest:

analyse.R
analysis.py