Weekend (tech) Project: Building CM11

I finally decided to dive into the deep end and compile Android from source!

This is not a complete how-to post. I'm writing this first to help you troubleshoot your own build and show where you might run into trouble. Primarily, though it's my notes for the next time I build.

Building:

First off, I'm building this in Ubuntu. Also, this is going to use some hard disk space. Make sure that at least 37 (more safely 40) GB is available. I didn't have quite this much available on my linux partition, so I plugged in my external HDD. You have to make sure that you have read-write-execute permissions to the mounted drive. The easiest way I found to do this is to use;

• sudo umount /media/madscientist/MyExternalHDDsudo mount -t ntfs-3g /dev/sd** /media/madscientist

You need to fill in the **. This can be found be finding where the device is located in /dev. You can do this with the command;

• sudo fdisk -l

From here on out, I'm following the instructions at;
http://wiki.cyanogenmod.org/w/Build_for_encore
with backup from this thread;
http://forum.xda-developers.com/showthread.php?t=1370873

Before you run envsetup and breakfast, you need to add a file, here's the description:

Have you set up your local manifest file to use the NookieDevs repo? If not, here is one that will work. This goes in the file [source directory]/.repo/local_manifests/roomservice.xml
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project name="NookieDevs/android_device_bn_encore" path="device/bn/encore" remote="github" revision="cm-11.0" />
<project name="CyanogenMod/android_kernel_bn_encore" path="kernel/bn/encore" remote="github" revision="encore-omap3-3.0.y" />
</manifest>

After this, you may need to re-run repo sync.

Okay, now my nook isn’t showing up on usb. Trying to find adb_usb.ini to edit as per;
http://wiki.cyanogenmod.org/w/Doc:_adb_intro#.22Device_not_found.22_errors
Finally, I found the file in; ~/.android/adb_usb.ini and could add the code: 0x2080

With this done, I restarted the adb server (adb kill-server, adb start-server).
You'll have to answer a dialog on the nook to confirm that the computer is allowed to run usb debugging. With this done I was able to attach to the debugger.

(Another useful site for nook usb info is here; http://nookdevs.com/NookColor_USB_ADB)

This allowed me to run;

• ~/Build/android/system/device/bn/encore$ ./extract-files.sh

Which requires that you already have cm 10.2 or later installed on the nook. This pulls some hardware information from the nook for building.

Then you can run the build as normal. From here on out I didn't have a problem building.

Installing:

Before you install, you ABSOLUTELY MUST back up your existing ROM. This makes it so very much easier to get back to a working state if something goes wrong. Like having an out-of-date recovery image....

When installing the build I encountered a “Build failed status 7” as per this thread
http://forum.xda-developers.com/showpost.php?p=47672324&postcount=353

This was because the recovery image was out of date. To install the latest recovery image, I followed the directions from http://nookdevs.com/CWR_Manual_Installation in which you’ll need to mount and copy some files over via adb. Here's the dump of my command line history:

• adb shell mount -o rw,remount /
• adb shell mkdir /boot
• adb shell mount /dev/block/mmcblk0p1 /boot
• adb shell mv /boot/uRecImg /boot/uRecImg.stock
• adb shell mv /boot/uRecRam /boot/uRecRam.stock
• adb push /buildpath/android/system/out/target/product/encore/kernel /boot/uRecImg
• adb push /buildpath/android/system/out/target/product/encore/ramdisk-recovery.ub /boot/uRecRam

This brought me to CWM-based recovery v6.0.4.6, which looks promising as google seems to associate this with CM11.

This worked - CM11 was up and running!

Grab the latest gapps from CM’s website (http://wiki.cyanogenmod.org/w/Google_Apps) and you're done!

Home Server - Part IV - Networked Printing

  Okay, so the Unison file sync is taking longer than I expected, but I realized that I could network the printer, so as to have any computer in the house print through the wireless. There were a few hiccups, but it was mostly painless. Here's what I had to do.

Part I - Installing Linux
Part II - Setting up SSH
Part III - Setting up SVN
Part IV - Networked Printing

  First you'll want to install the samba print driver in Ubuntu, instead of using the command line to install this time, I went to the software center, searched for samba, and installed from there. I followed the steps from this Ubuntu community link, but had to make some modifications as I went. It was fairly fast, and I had to modify most every step, so here goes. First off all the setup is Ubuntu server with attached printer (HP Officejet 5500), two Windows 7 boxes, and a Win XP laptop that needed to print to the printer.

  I went to http://localhost:631 from a browser on the Ubuntu box, but this didn't have much information on sharing the printer beyond allowing me to confirm the name. Next up was to open the samba configuration file "sudo vi /etc/samba/smb.conf". Under [printers] (there is something like #printers above it - don't be fooled, keep scrolling), change / add the lines to;
    browsable = yes
    guest ok = yes
    use client driver = yes
The last one is for the XP machine. Now restart samba with;
    sudo restart smbd
    sudo restart nmbd
  For the the Windows client printers, add the network printer by going to; Start -> Printers and Faxes -> Add a printer -> Network printer -> Now enter the local IP address of the machine and then the exact printer name for instance "\\192.168.1.21\officejet-5500-series". It will then give you a warning about installing drivers from the network, but then it will fail to install the driver. Select one close to you model (if you can find the exact model go for it).
  At this point, the Windows 7 machines were printing fine. However the Windows XP box was not. I had to go back into printers and faxes, then right click on the Network Printer -> Properties -> Advanced. If you were not able to select the exact driver when you set up the printer (as I was) this may help. Previously I had already used the printer from the XP machine, and it had correctly installed the driver then. I was able to pull down the menu next to driver and find the correct one. After clicking Apply, Windows prompted me to install the driver again, but (again) they didn't have it, so I canceled out of that dialog, and hit OK to the printer's properties. With the correct driver associated, the printer was printing! As always questions and comments are welcome!

Home Server - Part III - SVN

  Now that we have our linux box running, and we can connect to it from inside and outside the network, let's get to the good stuff - keeping all of our files in sync, and our version tracking the code. This allows us to always roll back if we should make a goof-up.

Part I - Installing Linux
Part II - Setting up SSH
Part III - Setting up SVN

  Some reminders from the previous cases - our example server sits on internal network at 192.168.1.21, our DNS service forwards traffic from the example domain madscientistlair.blogspot.com, and our SSH is configured to use example port 1986. We've so far been able to use PuTTY to connect.

  First off, we want to create a directory the non-administrator users can access (since we're using them when we SSH in). As the computer administrator we're going to create some directories, and then a usergroup, and finally add the users to that group, and give the group access to the folder. Then we can restrict SSH logins to those users, but they can still access all the files we want to access anyway.

  I chose to make a new high level directory /share with the command "mkdir /share". Within that I have /share/music, /share/photos, and /share/repos. This last one is where our subversion repositories will sit. The repository is where subversion maintains all of the files and file history that it uses. We will actually add the repositories first, and then do our group permissions to make sure that the permissions take on the repository files.

  I followed the quick start guide at the end of the svnbook to get myself started. First, we tell svn we want to use to repos folder as our repository;

    svn create repos

Now we add files

    svn import /tmp/myproject file:///var/svn/repos/ProjectA -m "initial import" 

The argument goes svn import [folder holding your files] [repository you want to add the files to]. The -m tells it what comment to associate with this version. It's always good to add comments to your commits so you can figure where to roll back to.

  Now I promised user groups and access to that folder. Create the usergroup and add users with the commands;

    sudo addgroup sharegrp

    sudo adduser username sharegrp

  Obviously username needs to be changed to the actual user name you want to add.
  Now we need to change permissions on the folder we created.
    chgrp sharegrp -R /share
    sudo chmod -R 775 /share
  This sets the group associated with the share folder as sharegrp. The second command changes the folder permissions to read-write-execute for owner (root) and group (sharegrp) and read-execute for any other users. The -R flag tells it to recursively do this for all the folders and files in the folder. I still had some issues being able to read / write to the folder immediately after, but a simple log-out and log-in fixed this for me.


  Now we've established and populated the subversion repository, we can check it out on our remote computers. I was following this guide, but the basic steps are to install TortoiseSVN. TortoiseSVN integrates into the windows shell, so that when you right click on or in a folder, you should see some new options "SVN Checkout" and "TortoiseSVN" with an arrow for more options. First we need to tell T.SVN that we'll be using ssh. So right click anywhere and go to TortoiseSVN -> Settings -> Network. Browse for the TortoisePlink SSH client. Mine was located at "C:\Program Files\TortoiseSVN\bin\TortoisePlink.exe". Now let's checkout the repository. Navigate where you want it, right click and hit "SVN Checkout". It will prompt us for the repository we wish to checkout. You want to end up with something like;
    svn+ssh://[username]@[PuTTY Saved session name]/share/repos/ProjectA
  Oh, I should mention; once PuTTY can connect to your server, re-enter the information, then give it a name on the main screen, and click save. This will save the session, and allow other programs, like plink or tortoiseplink (both command line interfaces to putty) to access this session. That's what I referred to above as the PuTTY saved session name. Also, it's useful to have Paegent running in the background during this, to run the authentication without prompting you for it. Click "Okay" and you should be on your way to checking out your repository!

  I also had to configure svn for my linux work machine. The command line interface is a tad bit different. First off, since we're using a non-standard port, we have to add a new ssh tunnel for subversion. Edit ~/.subversion/config under the [tunnels] heading add in myssh = ssh -p 422
Now when you go to checkout tell svn to use that tunnel with the command;

    svn checkout svn+myssh://[username]@[madscientistlair.blogspot.com]/share/repos/ProjectA

Now onto using subversion. The basic work cycle goes something like this;
    update -> resolve conflicts -> make changes -> commit changes
In TortoiseSVN click the folder you want to update and go "SVN Update". From the linux command line, either use svn update if you're in the folder, or svn update [path to folder] to update the specific repository. I've not yet had to resolve many conflicts, so I'm going to leave this blank for now. Once I have some more experience I'll link to a detailed discussion. When you're done commit the changes with svn commit -m "comment on the changes". Or use Tortoise, right click the folder and use SVN Commit. I have noticed that TortoiseSVN someitmes won't commit any changes, if you right click on the top level directory, and you have not checked out all the sub-directories.
  On this note, you may only want to check out part of a repository, here's how to do it. For instance, assume our project repository looks like;

  ProjectA

    |-Web

    |-Template

    |-Android

    `-Matlab

and you don't want the Web or Template folders checked out. When you initially check out the repository, select "Checkout Depth -> Immediate children including folders". This will make empty folders which you can then populate by right clicking and using SVN Update, but under options pull down "Fully Recursive" instead of "Working Copy".
  Again, when you're done making your changes, bring the server back up to date by right clicking and going SVN Commit.

  That's it. This is pretty cursory guide to getting SVN running on our server, but you should have a functional, if rudimentary, SVN system now.

Home Server - Part II - SSH

Now that we've gotten our Ubuntu server box up and running, it's time to install the tools that will allow us to connect, known as secure shell, or SSH.

Part I - Installing Linux
Part II - Setting up SSH

  First up is the install. Bring up a terminal, and use the command "sudo apt-get openssh-server" to install the SSH server. This is the command for Ubuntu, for Fedora, I think it's "yum openssh-server". You may also need to install the client. To check, first issue the command "man ssh" to bring up the manual page (use q to quit), if nothing comes up, then you don't have it installed, "sudo apt-get openssh-client" will do the trick.

  That's all for the installation, now let's test out the SSH capabilities. Behind the scenes, after you did the install Ubuntu starts up the ssh service. Later we'll make changes to the ssh files and will need to restart the service, so this will become more clear.
  My other computer is a Win XP box (good lord!), and I'm used to PuTTY to connect. We'll start on the local network. Grab the local ip off of the server with the terminal command "ifconfig". It returns a lot of other stuff, but somewhere in there should the be local ip address which looks like "192.168.#.##". Later I'll use the concrete example "192.168.1.21" as the host server. Load up a session of PuTTY and type in this address in the Host Name box, then click on Open. A command window opens that should be prompting you for your username and then password. You can use any of your accounts to test it out, after logging in it will drop you to the Ubuntu command line.

   With that working, let's make it a little more secure. Why do we want to make this secure? Here's a great article by the Fedora folks on security precautions and why you want to use them. I particularly liked the examples of the number of attempts to log in as root.
  The easiest things you should be able to do are 1) use a strong password on root, 2) change your port number to something non-standard, 3) use SSH 2, 4) deny root the ability to log in, and 5) use key authentication. I would recommend you implement all of Fedora's guidelines (if applicable), but I'll cover these four in more depth.

  Assuming you can take care of #1 on your own, here's how to handle number's 2-4. On the linux box, use the command "sudo vi etc/ssh/sshd_config". This opens up a text editor (called vi, see here for a list of commands) and shows the preferences file for the ssh server. Find and edit the text lines;

  Port 22 -> Port NNNN
  Protocol 2,1 -> Protocol 2

  PermitRootLogin yes -> PermitRootLogin no

  The right side of the arrow is what it should read when you're done. For port numbers, the most common are; 22 (SSH) 2222 (alternative SSH), 80 (webserver) 8080 (alternative webserver). So we'll pick an unused port say 1986. You can check it against this list, but it's not required that you pick something not on there, the max you can choose is 65535 (I think). Now if you try to use PuTTY to log-in again, you can't. We have to specify the port as well, the field directly right of the Host Name. With this you should be back to logging in just fine.

  Now we'll set up key authentication, replacing the user-id and password scheme. This makes you system less vulnerable to brute force attempts. Instead of trying a common name "root" and a bunch of passwords "", "password", etc. SSH will confirm you identity with a public/private key pair. This is actually how SSH transmits all it's data anyway, so your login will be as secure as the channel (or your computer). What happens in this scheme, is that you generate two "keys" one is called the private key - only you have the private key, and it identifies exactly who you are. The second is the public key, which you can give out to anyone! If you want someone to know that it is you at the other end, you encrypt a message (say "Hello world") with your private key. You send it to them, and they decrypt it with the public key you gave them, they see "Hello world" and know it's you. Why? Because if someone else had encrypted the message with her private key, gibberish would result on the output. Moreover, knowledge of the public key doesn't let someone know what your private key is. So as long as your private key is private, you can encrypt messages with it, and if they're decipherable, then the other party knows it is you at the other end. Of course there are drawbacks to this method - you could compute every possible private/public key pair, and then if someone knows your public key, they also have your private key. But if you make the keys longer, then it becomes much more difficult to compute every pair, and the system remains secure.

  I found this link extremely helpful in setting up key authentication using PuTTY and won't try to reproduce his work here; http://www.howtoforge.com/ssh_key_based_logins_putty_p2. Also be sure to encrypt (password protect) your private key, as if anyone gets their hands on your private key file they could impersonate you. Password protecting it gives you another layer of defense. For linux, you can use the command "ssh-keygen -t rsa" to generate the key pairs. Same steps apply for registering your public key with the server.
  Once you have key based logins on all of your computers, we can turn off username / password login. Open your config file again; "sudo vi etc/ssh/sshd_config", and change the following lines;
   ChallengeResponseAuthentication no
   PasswordAuthentication no
   UsePAM no

  So all this allows us to ssh into our server while on the same network, but what if we want to get to our files while on a different network, a friend's house or at work? Nowadays most IP addresses dynamically assigned and recycled, so we'll use Dynamic DNS (domain name system) to create a easy to remember web address for our server. I chose to use the FreeDNS service, although others exist, like OpenDNS and DynDNS is another popular one. For example, you sign up for a sub-domain, the "madscientistlair" part of "madscientistlair.blogspot.com". Then you use as the host name field in PuTTY, "madscientistlair.blogspot.com" and the DNS service says "Oh, you really want; 245.324.3.564", which is the current IP address your ISP has given your home. PuTTY connects to this instead, which goes to your router, which knows that requests for port 1986 should be sent to "192.168.1.21" and you connection to your server is made.
  The second part is installing a client to keep the Dynamic DNS servers updated. On my Ubuntu box, I used ddclient, installing with "sudo apt-get install ddclient". On setup, it prompts you for the service you're using. For more information check out the Ubuntu community page. Your router may support one of these services, go to your router web administration tools and then there's usually a Dynamic DNS tab.

  One other important step, and you may have noticed this above. How does the router know that you want to talk to the server computer? First it's useful to set a static IP for the server, this way the address "192.168.1.21" that we've been connecting to from home doesn't change in the middle of the night and we can no longer get to our server. The second is to set up port forwarding. There are usually tabs for both in the web administration tools for your router. Since these vary wildly, I can't give you any more advice other than on the port forwarding page, you want to have 1986 (or whatever SSH port number you specified in the sshd_config file) point to your server; 192.168.1.21. To check that it's working, go to http://canyouseeme.org/ and give it your port number. Below their ad, you'll see the result.

  There was one file issue I ran into. I could use "madscientistlair.blogspot.com" to talk to my server when I was outside of the house, but not when I was in the house! It turns out that some classes of routers don't allow you to do this. Essentially they prevent you from talking to a local machine through an outside connection. Kinda like using your cellphone to call you grandma in Florida so she can call the home phone. To get around this you'll have to have a batch script on your desktop that edits your windows "hosts" file when you switch locations. For example, you could have two scripts "set_when_home.bat" and "set_when_away.bat". You run set_when_home, when you're home, and this tells windows that when PuTTY asks to connect to "madscientistlair.blogspot.com" it really wants to connect to "192.168.3.564". When you're away, you run set_when_away, and it undoes this change. See here for a couple of example hosts files.

  Well that's it. We've installed SSH into our server, secured it, and allowed access from the outside. If you have any lingering questions, or can't get something to work, leave a comment and I'll try to help.

Matlab Startup (.m)

For those of you who use MATLAB on a regular basis, I'd like to share a trick I use to simplify my life considerably. One of the things I find myself constantly doing is resizing text on figures. I'm always increasing text to 14-16 point font and setting the face to bold. This is okay if you have one figure to make, but when you need to do it to every figure, and there are 10-20 staring back at you?

First you could start by setting the default figure - you can address its properties using 0 (zero) as the figure handle (see below for more on handles). But there's an even easier way! Using startup.m, we can set the defaults so that when MATLAB starts, the font size / face for the graphs will be set. When MATLAB loads, it looks for a script called "startup.m" which it will then execute before giving you control. By placing our overrides for the default figures here, we don't have to enter them in by hand every time we start MATLAB.

Here's how to do it: Open a new script and save it as startup.m in one of the paths MATLAB searches. You can find the paths by going to File -> Set Path. Something like "My Documents\MATLAB\" would be an example path for a Windows user.

Now for the contents of the script. Here's mine;

% startup.m
% Called at MATLAB startup.
% Sets defaults for figures (use 0 for figure handle)
% Adjust figure defaults for better font read-ability
set(0,'DefaultFigureColor',     'white',...
      'DefaultAxesColor',       'white',...
      'DefaultTextFontSize',    16,...
      'DefaultAxesFontSize',    14,...
      'DefaultTextFontWeight',  'bold',...
      'DefaultAxesFontWeight',  'bold');

% Load up my lovely custom colormap that uses black as middle value.
load('MyColormaps');
set(0,'DefaultFigureColormap',mycmap);
clear;

Let's go over each command;

set(0,'DefaultFigureColor', 'white',...,'DefaultTextFontSize',16,'DefaultAxesFontSize',14,...);

If you're not familiar with the get() and set() commands you really should be. These are a great instant-reference when you need something. The syntax is usually get(FigureHandle) and set(FigureHandle,Property,Value). If you're working on only one figure, you can use get(gcf) (get all values for the current figure), a list of properties and values will be displayed to you. If you are working with multiple figures, it is usually best to assign each one a unique handle, so you don't get confused. For example; "myh1 = figure;" assigns the next figure created a handle of "myh1", then you can use get(myh1) and set(myh1,...) to get/set values of that figure.

Now the second group of commands;

load('MyColormaps');
set(0,'DefaultFigureColormap',mycmap);
clear;

This loads a file called "MyColormaps" within which I have saved a colormap, "mycmap" (creative, I know). You don't need this unless you are (like I am) highly unsatisfied with the default colormaps. I'll go over in the future how to make custom colormaps, but you can leave that out of your startup.m for now. Finally, I clear the variables "MyColormaps" has loaded in my workspace (I like having an empty workspace on startup).

I'm always interested in hearing about your tips and tricks when it comes to simplifying life in MATLAB / Mathematica, etc. Give a shout back if you have something you'd like to share.