Wednesday, April 24, 2013

Git tag and versioning your project

We use tags to version our project. I will set up here basic commands we use on everyday basis and may use to handle some issues.

Showing your tags

Listing the available tags in Git is straightforward. Just type git tag.
$ git tag
1.0.7
1.0.8
1.0.9
1.1.0

Adding tags

To add tags you may do git tag tagname. But better to specify args to be able to use in scripts:
$ git tag -a 1.1.1 -m "major improvements"
$ git tag
1.0.7
1.0.8
1.0.9
1.1.0
1.1.1

Here we have added a tag version 1.1.1 with commit message "major improvements".

Uploading to repository

I assume you have a GitHub repo. So to push your tags there run git push (for your tagged commits commits) and then push your tags:

$ git push
Counting objects: 120, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (59/59), done.
Writing objects: 100% (69/69), 390.70 KiB, done.
Total 69 (delta 41), reused 0 (delta 0)
To ****************.git
   ******..******  master -> master
$ git push --tags
Counting objects: 1, done.
Writing objects: 100% (1/1), 157 bytes, done.
Total 1 (delta 0), reused 0 (delta 0)
To ****************.git
 * [new tag]         1.1.1 -> major improvements


You may now check this is pushed by clicking your GitHub repo's Tags button.

Removing a Tag

In case you have pushed a wrong tag... For example, if you have created a tag called push in a Git repository you would remove it from your repository by doing the following:
git tag -d push
git push origin :refs/tags/push



Now it's up to you to handle it in a release/deployment/staging/whatever script. Hope it helps. Besides, this should handle 99% of your tasks with tags. If not feel free to use official docs ;)
Comments and suggestions are welcome.

Thursday, April 4, 2013

Django 1.5 migrating to new url tag syntax

Django 1.5 changed. The url tag syntax from {% url  my_page %} into {% url "my_page" %}. Now all my templates should be changed accordingly to migrate to 1.5. You could do it with executing this command in your bash shell.
Assuming you are in the project folder. Assuming you have version control, or do know how to backup your work ;). Some people claim it is unsafe. But you may want to make it safer with adding '{%' to the beginning of the string. To make sure only templates arre changed. Or either execute this only in your templates folder to be sure.
$ find . -type f -print0 | xargs -0 sed -i 's/ url \([^" >][^ >]*\)/ url "\1"/g'
And one person did a django snippet for this operation. You may also find it useful. http://djangosnippets.org/snippets/2905/

Sunday, February 17, 2013

Automating everyday system routine with Fabric (Python)


Lets talk about your console work. I use it on everyday basis. I need to log on to  my deployment/stage/whatever, server and do some redundancy. E.g. download some logs, clean up some caches and/or redeploy something. Here is the occasion, when Fabric comes handy. You can eliminate all the redundancy and cover tons of operations you need to do every day, using console, shortcutting them to one of your simple commands, like: fab deploy_production -H root@mydeployment.com interested? Let's move on then. Notie this command is not a masterpiece, but must give you the understanding of usual workflow.

1. Installation and purposes

Let's go ahead, installing Fabric and automating some simple operations. You can read official docs about alternative methods. But installation is fairly simple. And is the matter of typing:

$pip install fabric

Nothing more special is required. You will need fabric at system wide scale. I install it locally with superuser privilegies. Note I'm assuming installation of this on my MacBook and not on a production server or any remote destination. Fabric really is a shell wrapper with commands queue that executes your console tasks. Something similar to Django unit tests, if you know what I mean. E.g. Developing a fabric script is a process of copy-paste your console interactions and here you go... Similar to BASH/SH scripts. But why use fabric then?

How would you handle commands exit status in shell or bash? Write some error handling right? And this one comes out of the box with it ;)
Let's not just talk, but write some

2. Example usage

To write your commands you need to make a file called fabfile.py with syntax that is described in fabric documentation. But it's almost "pure" python. So everything must be understood easily.

Fabric supports all the python stuff. And good IMHO practice it to split your task to steps and execute them one by one. You can use def method to define them. E.g.:

"""Test script for article"""

from fabric.api import run

def set_up():
    """does some setup automation"""
    run('setup something')

def run():
    """main run method, that now calls set_up() method"""
    set_up()

Now you can run a command fab --list in this directory. Output will be like so. Note how it converted all your comments into handy help text explanation for your script.

leopard-2:article garmon$ fab --list
Test script for article

Available commands:

    run     main run method, that now calls set_up() method
    set_up  does some setup automation
leopard-2:article garmon$

Now we have got a handy script to setup something. But let's not get away with it and show you some real life example. It will also be converted to hide some (possible) sensitive information...

3. Real life example


"""Example script for article with deployment parts"""

from fabric.api import run, sudo, cd, get, local, lcd, prefix

def cleanup_environment(folder_path):
    """ deletes old deployment instance"""
    sudo('rm -rf %s' % folder_path)

def install(folder_path):
    """ Installs an instance of my website into production deployment server with recreating a virtual environment"""
    with cd(folder_path):
        sudo('virtualenv ve --no-site-packages')
    with prefix('source %sve/bin/activate' % folder_path):
        sudo('pip install mysite')

def restart():
    """Restarts a service of my website"""
    sudo('service mysite restart')

def stop():
    """Stops a service of my website"""
    sudo('service mysite stop')

def run():
    """Runs a service of my website"""
    sudo('service mysite run')

def download_logs(folder_path):
    """ Download logs files and db from server to local machine current directory
        This will download all the logs from deployment server
        into your current directory (locally) with current timestamp"""
    import datetime
    time_stamp = datetime.datetime.strftime(datetime.datetime.now(), "%Y-%m-%d-%HH-%MM-%SS")
    with cd(folder_path):
        sudo('tar -czvf /tmp/dms_logs.tgz ./log/dms.log*', user='user')
    local('mkdir ./%s/' % time_stamp)
    with lcd(time_stamp):
        get("/tmp/dms_logs.tgz", "./dms_logs.tgz")
        local("tar -xzvf dms_logs.tgz")
        local("rm -f dms_logs.tgz")
    sudo("rm -f /tmp/dms_logs.tgz")

def main():
    """Redeploy your Django powered website"""
    deployment_path = '/srv/www/django'
    download_logs(deployment_path)
    stop()
    cleanup_environment(deployment_path)
    install()
    run()

This script has main commands for me to run make redeployment of my website into a production web server, with archiving all the data (logs in this example) into a directory with current timestamp.

You must run it with a parameter -H specified. this parameter represents a host that our little script will ssh to. and is something like username@192.168.1.1, or something you can ssh to, using command ssh. E.g. -H stage-server in case you can locally type ssh stage-server. So the full command to run this will be something like: fab main -H my_deployment_server

Most fo the code in this script is given as an example. And may contain some logical errors, because it is taken from different parts of a real life deployment commands. But I really hope it will help you to master your own deployment with this handy tool.

Comments and suggestions are welcome...

Friday, January 4, 2013

Raspberry Pi boot applications Autorun

I had a problem with running required programs upon system startup. I had to set up hdparm utility each time system boots up. I have 2 external HDD's connected and require setting their sleep time for 10 minutes each boot. So setting this up. The answer is found at Debian administration guides.  I'll provide it in the end of the article. Here is my decision based on that:

So to set up a program to run on system boot. (In fact one of the system run-levels). You can add it's name. But a good practice will be to add an sh script with execution of this utility an all the parameters.
Sample script is:

#! /bin/sh
# /etc/init.d/blah
#

# Some things that run always
touch /var/lock/blah

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    echo "Starting script blah "
    echo "Could do more here"
    ;;
  stop)
    echo "Stopping script blah"
    echo "Could do more here"
    ;;
  *)
    echo "Usage: /etc/init.d/blah {start|stop}"
    exit 1
    ;;
esac

exit 0

You can insert your utility name instead of "blah" and give it a spin. My script looked like:
And make sure it is owned by root user:

sudo chmod 755 /etc/init.d/blah
sudo chown root:root /etc/init.d/blah

So my script began to look like:

#! /bin/sh
# /etc/init.d/hdparams
#

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    echo "Setting Connected HDD sleep timeout to 10 Minutes."
    hdparm -S 120 /dev/sda1
    hdparm -S 120 /dev/sdb1
    ;;
  stop)
    ;;
  *)
    echo "Usage: /etc/init.d/hdparams {start|stop}"
    exit 1
    ;;
esac

exit 0

Because I really don't care about hdd spin down time at shut down and/or system reboot.
Original article is: http://www.debian-administration.org/articles/28
Hope it helped somebody. Please comment if you used/found some inconsistency...

Sunday, December 30, 2012

Raspbery Pi as a home file server

I've wanted more from my PI, besides Time Machine functions, established in my earlier articles. So I've decided to make it a SAMBA server. I have a Mac and a Windows PC. And several iPads/iPhones. So the decision was obvious. To access my storage content I'd need a samba share. Because it is recognized by all this tech. iPad has Oplayer, to watch movies without conversion... GoodReader for docs. One word - SAMBA share. SO this article will be about adding a samba share to your Raspberry Pi.
Note I have a Time Machine already set up. But it won't matter much. I only have another drive index letters and so on. I'll try to cover this as much as possible. And macs somewhat easily read NTFS filesystems. So we will be mounting NTFS filesystem volume to a Raspberry Pi with SAMBA network sharing. Let's get on to it.

To set up a Raspbery Pi home network Samba server you will need to:
- Connect an external drive (USB HDD in my case).
- Update fstab for auto-mounting of it on system boot
- Install and configure samba
- Enjoy

1. Connecting an external USB drive.

Assuming you hava an external hard drive connected type the command:
pi@raspberrypi /mnt $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" UUID="XXXX_XXXX" TYPE="vfat" 
/dev/mmcblk0p2: UUID="XXXXXX_XXXXXX_XXXXX_XXXX" TYPE="ext4" 
/dev/sda1: UUID="XXXXXXX-XXXXXXX-XXXXXXX-XXXXX" LABEL="Time Machine" TYPE="hfsplus" 
/dev/sdb1: LABEL="Data1" UUID="5XXXXXXXXXX1" TYPE="ntfs"
Note I have sda1 drive with hfsplus filesystem for AFP shares so my NTFS drive is named sdb1. Yours will probably be sda1 out of the box. Note that. We now see the NTFS hdd. Lets make it mounted. Typing mount command will show it's not yet there.
pi@raspberrypi /mnt $ mount
/dev/root on / type ext4 (rw,noatime,user_xattr,barrier=1,data=ordered)
# ...
/dev/sda1 on /mnt/TimeMachine type hfsplus (rw,nosuid,nodev,noexec,relatime,umask=0,uid=0,gid=0,nls=utf8)

2. Update fstab for auto-mounting of it on system boot

Lets make a directory for mounting a drive:
pi@raspberrypi /mnt $ sudo mkdir /mnt/data
Now lets update fstab to add a setting for proper auto-mount of our drive on boot.
pi@raspberrypi /mnt $ sudo nano /etc/fstab
Lets add a line. Something like.
UUID="XXXXXXXXXXXXXX" /mnt/data ntfs rw,auto 0 0
Where UUID is your device UUID, that you can find out using sudo blkid command.
Now we can either reboot or mount a hard drive. I choose to mount. Lets type:
pi@raspberrypi /mnt $ sudo mount /dev/sdb1
pi@raspberrypi /mnt $ mount
/dev/root on / type ext4 (rw,noatime,user_xattr,barrier=1,data=ordered)
# .....
/dev/sdb1 on /mnt/data type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096)
And our drive is mounted and will be auto-mounted on each reboot.

3. Install samba

Now lets move on to installing samba server for our network sharing. Lets do it with apt-get:
pi@raspberrypi /mnt $ sudo apt-get install samba
Confirm additional package downloads by typing y to the prompt of this command. And wait while it installs some demons and defaults.
Now lets edit the default samba config:
pi@raspberrypi /mnt $ sudo nano /etc/samba/smb.conf
And add the config of our drive to the end of this config file. (After numerous other parameters):
[Data]
comment = DataDrive
read only = no
locking = no
path = /mnt/data     
guest ok = yes
force user = pi
After this is done simple restart of samba server will be sufficient:

pi@raspberrypi~$ sudo service samba restart

4. Enjoy

Now I have a file dump server I ever dreamed of. I can connect as guest with all my devices and have a cross platform decision built on a PI platform.
And this all stuff looks like this now. With a second ntfs external HDD and my Pi in Punnet MK1 paper case.

Tuesday, November 27, 2012

Raspbery Pi Time Capsule HDD managing power saving mode

I want to sugest power saving changes after installing and handling creation of Time Capsule for Time Machine for my home network (form my previous article). I like when it's quiet. And like when disks does not spin, when it's not necessary to do so. So if you usually backup once in some hours and main time (Night and a part of the day) you do not touch a Time Machine... You would probably want to slow down your disks for that time. Fortunately Linux does it very well. And support of it is encoded into Linux core (kernel). You just would need a UI for this possibilities and a bit of knowledge and luck ;)

Warning: Operations described farther may be potentially hazardous to your hardware.

So now after you are warned ;) let's continue. I guess I will not touch those hazardous functions, but you may, typing wrong letters to this command in Linux shell... So be careful. Better read manuals for farther commands and so on...

Anyway what we need is a UI. And we have it in form of nice utility called hdparm. It is not included in current Raspbian distribution (and it should not be perhaps) so you would probably need to install it by yourself. In fact you can find a package with it. But I'd recommend installing from latest version. It is http://hdparm.sourceforge.net/ situated. You can download it there. Using wget, for e.g. So move on to downloading and unpacking it. If you have a default Raspbian kernel, or did not screw up something useful while configuring and compiling your on one... You would probably simply enter a directory and type something like this:
pi@raspberrypi ~ $ wget http://switch.dl.sourceforge.net/project/hdparm/hdparm/hdparm-9.43.tar.gz
pi@raspberrypi ~ $ tar -zxvf hdparm-9.43.tar.gz
pi@raspberrypi ~ $ cd hdparm-9.43
pi@raspberrypi ~/hdparm-9.43 $ ./configure
pi@raspberrypi ~/hdparm-9.43 $ make
pi@raspberrypi ~/hdparm-9.43 $ sudo make install
pi@raspberrypi ~/hdparm-9.43 $ hdparm

hdparm - get/set hard disk parameters - version v9.43, by Mark Lord.

Usage:  hdparm  [options] [device ...]

Options:
...
Hurray! we have our desired utility. Reading through it's help you would probably find a useful for us paramether:  -S   Set standby (spindown) timeout
Here is our goal. I've set mine to 120 (10 minutes) by executing:
pi@raspberrypi ~ $ sudo hdparm -S 120 /dev/sda1

/dev/sda1:
 setting standby to 120 (10 minutes)
pi@raspberrypi ~ $
I have a WD "green" drive and will benefit from this. But you may harm your drive with often standbys and spinups. So choose wisely and read man hdparm for more ;).
Hope you would benefit from my finding. Please comment in either way.

UPD: to persist this upon boots see article:
http://garmoncheg.blogspot.com/2013/01/raspberry-pi-boot-applications-autorun.html

Time Capsule for $25

The real article name might be something like: Configuring Raspbery Pi to serve like a Time Capsule with Netatalk 3.0 for Mountain Lion.  But it's too long ;)

Here I will describe the process of using Raspberry Pi like a Time Machine in my network. To be able to backup your MAC's remotely (Like it would be NAS of some kind). It assumes you have a Raspberry Pi and have installed a Raspbian there and have a ssh connection, or somehow having access to it's console. Refer to my previous article for details.

Now that we have a Pi that is ready for action let's animate it. So to make it suit you as a Time Capsule (NAS) for your MAC's you need to do those basic steps:
- connect and configure USB hard drive(s)
- install support of HFS+ filesystem to be able to use MAC's native filesystem
- make mount (auto-mount on boot) of your hard drive
- install Avahi and Netatalk demons
- configure Netatalk daemon to make it all serve as a Time Machine
- configure avahi demon
- put avahi and netatalk demons into autolaunch on system boot
- ENJOY.

1. Setup a Hard drive.

I assume you know what you are doing and understand you have to have a hard drive formatted under your mac's HFS+ filesystem. That you, like me, used as an external hard drive for backups before this. Just for example. Or bought it... Whatever. One word you will need (or assume you have) an external USB drive to follow me farther.

Start your drive (in case it has external power). I think Pi would handle only drives with external power. But who knows ;). And connect it to your Pi. Let it spin for some seconds to be recognized.
Now connect to console of your Pi and let's roll.
leopard$ ssh pi@192.168.1.140
pi@192.168.1.140's password: 
Linux raspberrypi 3.2.27+ #250 PREEMPT Thu Oct 18 19:03:02 BST 2012 armv6l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Mon Oct 29 01:04:39 2012
pi@raspberrypi ~ $
As you can see I have a clean system installed from latest (as for 24.11.2012) downloaded image.

Let's do some magic. Because Raspbian does not support Apple's filesystem out of the box. At least we need to mount it write enabled. Which is not set up by default too. SO let's start from typing command blkid under root and looking at output.
pi@raspberrypi ~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" UUID="8B12-9112" TYPE="vfat" 
/dev/mmcblk0p2: UUID="AAAAAAA-1111-1111-......." TYPE="ext4" 
/dev/sda1: UUID="AAAAAAA-1111-1111-......." LABEL="Time Machine" TYPE="hfsplus" 
/dev/sda2: LABEL="Data" UUID="1234567890......." TYPE="ntfs" 
pi@raspberrypi ~ $
Note UUID of your device. Mine is called sda1 and has a partition type hfsplus.  Hurray! Drive is connected and is recognized by Pi.

2. Install support of AFP filesystem (MAC's native).

Now let's install tools we need for HFS+ filesystem mounting. We need  to mount it RW to be able to backup.
pi@raspberrypi ~ $ sudo apt-get install hfsplus hfsutils hfsprogs
Press "y" in the process of install. You will get HFS+ support libraries installed. So let's move on to mounting.
After this you should be able to mount your HDD. You may try it or skip to making permanent mounting config via /etc/fstab (Step 3).
pi@raspberrypi ~ $ sudo mount -o force /dev/sda1 /mnt/TimeMachine
pi@raspberrypi ~ $ cd /mnt/TimeMachine
pi@raspberrypi /mnt/TimeMachine $ ls -l
total 116
drwxr-xr-x 1 root      99      6 Jan 11  2012 Backups.backupdb
drwxrwxrwx 1  501 dialout     31 Feb  1  2012 !DVD-s
-rwxr-xr-x 1 root root    115716 Sep  1 07:23 tmbootpicker.efi
pi@raspberrypi /mnt/TimeMachine $
You may see your files from a HDD. Hurray! Again ;). Btw you can look at how is your device was mounted by typing mount command into prompt. It will show your mounted volume and mode it was mounted. So you will see something like this:
pi@raspberrypi /mnt/TimeMachine $ mount
/dev/root on / type ext4 (rw,noatime,user_xattr,barrier=1,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=118872k,nr_inodes=29718,mode=755)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=23788k,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=47560k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=ascii,shortname=mixed,errors=remount-ro)
/dev/sda1 on /mnt/TimeMachine type hfsplus (rw,relatime,umask=22,uid=0,gid=0,nls=utf8)
Note last line with /dev/sda1 mounted in mode "rw". Yes everything works well. Let's move on.

3. Make mount (auto-mount on boot) of your hard drive

Let's create a directory for our new tome. We will need this to mount filesystem.
pi@raspberrypi ~ $ sudo mkdir /mnt/TimeMachine
Lets unmount our tome (in case you did tested it exists with me).
pi@raspberrypi ~ $ sudo umount /dev/sda1
And add a proper fstab entry to auto-mount this hard drive on boot. Open fstab:
pi@raspberrypi ~ $ sudo nano /etc/fstab
And add a line to the end of the file, indicating our Time Machine tome mounting preferences. Mine was:
UUID="AAAA-BBBB..." /mnt/TimeMachine hfsplus rw,force,exec,auto,users 0 3
Where AAAA-BBBB...  is your devise UUID from that blkid command remember?
I used UUID to be sure it won't matter if I'll switch hard drives places in USB ports. But you may use /dev/sdXX, where XX is your device counting numbers in /dev. Mine in this case was /dev/sda1.

Note: You can skip reboot.

Let's now do a reboot to make sure it all spins in automatic mode ;).
pi@raspberrypi ~ $ sudo reboot

Broadcast message from root@raspberrypi (pts/0) (Wed Nov 21 19:13:49 2012):
The system is going down for reboot NOW!
After system boots up run mount command to make sure we are ok in rw mode. Btw you may play with commands:
# to unmount volume:
pi@raspberrypi ~ $ sudo umount /dev/sda1
# to mount again:
pi@raspberrypi ~ $ sudo mount UUID="AAAA-BBBB..."
While you will find your string. BTW you can read about how and why in /etc/fstab manual here... Or google about it.

Note: if your HFS+ tome does not mount and/or mounted read only. You should try to run fsck for hfsplus partition type manually.
pi@raspberrypi ~ $ sudo fsck.hfsplus -f /dev/sda1
Anyway we must now have an rw mode HFS+ volume mounted and ready for our Time Machine. So Lets move on...

Checking your user permissions. As we will use user "pi" farther along the article we must add him ability to manage Time Machine data. (In case you are not mounting empty drive). You should consider changing all the data owner to user "pi" to make sure we will be abler to backup in future:
pi@raspberrypi ~ $ sudo chown -R pi /mnt/TimeMachine

4. Install and configure Avahi and Netatalk demons

First make sure you have the latest packages lists, running:
pi@raspberrypi ~ $ sudo apt-get update
And updating what necessary.

Now install all the required packages with db and encryption support in order Avahi and Netatalk demons to support for HFS+ filesystem:
pi@raspberrypi ~ $ sudo apt-get install avahi-daemon libavahi-client-dev libdb5.3-dev db-util db5.3-util libgcrypt11 libgcrypt11-dev
After this is complete we will have almost everything required for our little demon.

Download and unpack 3.0.0 (latest, as for now is 3.0.1 but I had problems with it spinnig) Netatalk demon sources. For e.g. using command: $ wget http://somesite.com/photos.zip You may get Netatalk at SourceForge. And unpack using e.g. tar: tar -xvf netatalk-3.0.0.tar.bz2

After you are ready with unpacked netatalk distribution go ahead and make custom configuration of it. Enter directory of unarchived netatalk and execute command:
pi@raspberrypi ~/netatalk-3.0 $ ./configure --with-init-style=debian --with-zeroconf
It will do the proper configuration for our needs. After this is done run:
pi@raspberrypi ~/netatalk-3.0 $ make
pi@raspberrypi ~/netatalk-3.0 $ sudo make install

5. Configure Netatalk daemon to make it all serve as a Time Machine

Now when you will have all this done go edit a configuration file of Netatalk. From version 3.0 it is located in /usr/local/etc and has Samba-like look. You can symlink this file to /etc/afp.conf if you wish... 
pi@raspberrypi ~ $ sudo nano /usr/local/etc/afp.conf
And insert this sample configuration. This config file works with latest edition ( now it is 10.8.2 ) of OS X Mountain Lion. So the config:
;
; Netatalk 3.x configuration file
;

[Global]
; Global server settings
; Name of your computer in apple devices network
hostname = TimeMachine Pi
; IP of your Pi
afp listen = 192.168.1.140
; logging config
log file = /var/log/netatalk.log
log level = default:info

[Homes]
basedir regex = /home
cnid scheme = dbd
; Display each user home directory in this format
home name = Home: $u
[Time Machine]
; Our Time Machine volume
path = /mnt/TimeMachine
cnid scheme = dbd
file perm = 0660
directory perm = 0770
time machine = yes

; Example to add a new static share:
; [My AFP Volume]
; path = /path/to/volume
It is quite well commented so you will be able to copy-paste and edit ;).

6. Configuring AVAHI services 

Now you would probably want to have a Time Capsule as a separate share in your network shares. Note this step is optional. But if you plan to use your pi for more than 1 thing, like I do... I'd recommend to do it. Create a special afpd services configuration file by executing:
pi@raspberrypi ~ $ sudo nano /etc/avahi/services/timecapsule_afpd.service
Then add this contents to file:
<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
      <name replace-wildcards="yes">TimeCapsule %h</name>
      <service>
          <type>_afpovertcp._tcp</type>
          <port>548</port>
      </service>
      <service>
          <type>_device-info._tcp</type>
          <port>0</port>
          <txt-record>model=TimeCapsule</txt-record>
      </service>
</service-group>
After this is configured for your desired settings. Go run your demons:
pi@raspberrypi ~ $ sudo /etc/init.d/netatalk start
pi@raspberrypi ~ $ sudo /etc/init.d/avahi-daemon start
Avahi demon was installed from package. So you might need to actually run restart instead of start command. Now go check your Mac's Network Shares.

7. Put avahi and netatalk demons into autolaunch on system boot

We have installed here an avahi demon from package. So it must be written to all system run-levels during installation. Only thing here is compiled Netatalk 3.0 so you can make a proper trick here by putting it to run with default init.d command:
pi@raspberrypi ~ $ cd /etc/init.d/
pi@raspberrypi ~ $ sudo update-rc.d netatalk defaults
update-rc.d: using dependency based boot sequencing
update-rc.d: warning: default stop runlevel arguments (0 1 6) do not match netatalk Default-Stop values (1)
pi@raspberrypi /etc/init.d $
You can read about system runlevels and why this command and not another in this article:
http://www.debuntu.org/how-to-manage-services-with-update-rc.d I find it nice simple and easy.
You can also read about Time Capsule theory and find answers to many "why?" questions here: http://buffalo.nas-central.org/wiki/Time_Machine_&_Time_Capsule_support_on_your_LinkStation

Result:

TADAM. The moment of truth. You can look into your Network Shares on a MAC. Select a virtual Time Capsule and hit "Connect as..." button. Login to your raspberry with user pi and saving password. Enter Time Machine directory... And Time Machine share will appear on a desktop (If it is configured to appear for newly mounted volumes...). You will experience something like this:
After all of this is done. Go enter your Time Machine settings and add a new drive. Bckuping on a time capsule is some kind of different to backing up into external (e.g. USB) hard disk drives. Using external HDD as a Time Machine backup disk will store folder Backups.backupdb into the root of your drive. But backup into time capsule (network backup) creates a volume like leopard.sparsebundle, where this leopard is your MAC's name. And connects it as a file system image, storing this folder inside. So to make initial backup connecting through USB and then continue making backups is possible but irrelevant. Because you would have to mount this image and change permission of all files. Then move them into mounted virtual image under Pi or over network. So the advantages of this idea are controversial. 

It was looking like so for me:
And my pi, serving like a Time Capsule and external HDD:

Please comment your thoughts about all of this. Every comment is appreciated. Thanks. And hope someone would benefit from those my findings.

Monday, November 19, 2012

Raspberry Pi first steps and basic network configuration on a Mac

Here are my first steps. And I hope you will find something useful here, while configuring your Pi...
First of all. Mine have been bought on ebay, from resellers. And were delivered a while ago. Main purpose of this purchase war to attach a headless server to my router. I have 2 external HDD's and would like to have torrents, Time Mashine fro my macs and so on. So buying a handheld computer like this would be a bargain for me.

As for built in decisions like some kinds of NAT devices and different routers with external HDD features... They are either cost a lot or lack some kinds of desired functionality. So the goal is to make some kind http/api manageable server in my local network with Time Mashine and file storage/backup. Just for fun. And to have only laptop on my work table.

SO back to the Pi. If you are buying "device only" configuration, like I did. First of all you'll need some different kind of things many computer fans usually have for backup purposes, or just in case... Those are device with HDMI port or either a TV set with AV input, e.g. SCART or Video in port. A keyboard (USB, of course) and SD card usually bigger then 2 Gigabytes.

I had this all. So I've connected my PI and tried to flash a card, having problems doing it. I'll try to cover most of them here.

1. Install a system. Usually it is a simple "burning an SD card" with Raspbian Linux distribution. To do this you need to download it from: HERE and do actions from official manual here with looking further note first.

Problem: "dd" does not work like intended, because volume is bisy.
Solution: We need to unmount volume, using diskutil command with "force" flag for e.g.:
sudo diskutil umount force /Volumes/my_flash_drive

Problem: Doing "dd" command under mac environment did write the card, but took ages on my mac's config. So the...
Solution: Is modified dd command call with manually specifying block size. This will speed up sd card write tremendously.
dd bs=1m if=~/Downloads/debian6-19-04-2012/debian6-19-04-2012.img of=/dev/rdisk1
Note "bs=1m" here. It helped my SD card burning not to hang out to infinite time.

After SD card was written and you are good to go...

2. Pi initial config. Appears after first boot and looks like so:
You can run it afterwards using command "sudo raspi-config". Just in case you would missed something hitting Esc or similar. It will be available via ssh in future also.

Tools you will use here are similar to bios config on a PC. So to get started:

Here you need to expand rootfs (to use entire SD card space) I doubt you have original Pi Raspbian card. But you may skip this step in case you do. This will be executed on next start. So feel free to do it first thing.

Next important option is change pass and you need it no-mater what. It will change your default user's with username pi password. It is similar to user with root permissions... So be sure to do it. Note password symbols are not displayed during entry.

change timezone goes next. If you do not leave in other timezone, like I do, change it. ;) Note pi does not have an internal "bios" clock so you would need to do it. Because time is used from internet atomic clocks...

memory split is also important option that help you to manage precious pi's RAM. I've used 16 because I will not need GUI and will run "headless", using it via ssh/network services. To use GUI consider 32+ Mb or RAM for video memory. But note RAM is really in lack for most modern services. and you may need every Kb of it in future.

ssh is our main goal. To access it/configure it in future. Hit it to enable ssh connections.

and you can run update if you have already connected network somehow :).

That's it with basic config for our needs. Here goes...

3. Static network configuration.
I think this is the most important part. Because I have googled a lot to find a proper solution for my needs.

Pi is connected via dhcp IP address, usually given randomly by your router. And you need to make it static to know you will always have pi under certain network address.

You may know your Pi's ip address by entring route command into prompt. Yo will see something like this.

pi@raspberrypi ~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
192.168.1.0     *               255.255.255.0   U     0      0        0 eth0


Here you can see your network is in range 192.168.1.x. Biut if you are configuring your Pi without actual network config, like I did, you will need to do something like this:

You need to know it from your router config. And you can simply connect it via network, if you did it properly in past. Or either connect, like you did for initial router config. You need something like this, saying about your LAN network address. I have a wired mix of different network decisions in my router. So note your typical network router address here:
Here you can know your router's DHCP address. And usually you can do it by examining your router's user manual.
Anyway this tells me that I have a wire (LAN) IP network with adress range 192.168.1.x where x is between 1 and 254. Your DHCP in router may work in another way and have different interface. So you may have other IP range set. E.g. Mine had a 100-199 range by default:
I've chosen a 192.168.1.140 IP  for my PI, because I would not have 40 devices in my network and would not have any possible IP conflicts this way.

This are data required for our Pi network config. We need to manually change this in our Pi's config files.

Try to do this:

cd /etc/network
sudo nano interfaces

This will start default Pi's text editor, called "nano", pointing to networks configuration file. Here is how it might look like:
You can notice nice hotkeys here in the bottom. So I think you will be pleased with this user interface. Let's replace the line iface eth0 inet dhcp with lines.

iface eth0 inet static
address 192.168.1.140
netmask 255.255.255.0
gateway 192.168.1.1

This will create a static address for our Pi. Now to make sure we have a proper DNS. Go to editing /etc/resolv.conf. E.g. type:

sudo nano /etc/resolv.conf

Mine had a a line:

nameserver 8.8.8.8

Pointing to google's DNS. And I've left it, adding my router as a first DNS. So my resolv.conf became looking like:

nameserver 192.168.1.1
nameserver 8.8.8.8

And that's it. After doing this you would have to restart network interfaces. Or better do "sudo reboot" to make sure everything is configured. At this point you are OK to go running headless already. You can now do ssh connections to static ip at your pi.

So to use it you would need only to type in your terminal:

ssh pi@192.168.1.140

This will connect you to your pi. And you are ready to go ahead.
you may see something like this:

And to finish here is how my Pi looks like in a "printable case" The Punnet from user "E". Thanks to him.