Mar 6, 2017

POP3 Mock (Fake) server using python script

Having a need in POP3 server for my debugging purposes I have used this script.
Letting it to be here in case of anyone would need to do something similar.
Usage is:
$ python 110 email_file.eml

"""pypopper: a file-based pop3 server

    python <port> <path_to_message_file>
import logging
import os
import socket
import sys
import traceback

logging.basicConfig(format="%(name)s %(levelname)s - %(message)s")
log = logging.getLogger("pypopper")

class ChatterboxConnection(object):
    END = "\r\n"
    def __init__(self, conn):
        self.conn = conn
    def __getattr__(self, name):
        return getattr(self.conn, name)
    def sendall(self, data, END=END):
        if len(data) < 50:
            log.debug("send: %r", data)
            log.debug("send: %r...", data[:50])
        data += END
    def recvall(self, END=END):
        data = []
        while True:
            chunk = self.conn.recv(4096)
            if END in chunk:
            if len(data) > 1:
                pair = data[-2] + data[-1]
                if END in pair:
                    data[-2] = pair[:pair.index(END)]
        log.debug("recv: %r", "".join(data))
        return "".join(data)

class Message(object):
    def __init__(self, filename):
        msg = open(filename, "r")
   = data =
            self.size = len(data)
  , bot = data.split("\r\n\r\n", 1)
   = bot.split("\r\n")

def handleUser(data, msg):
    return "+OK user accepted"

def handlePass(data, msg):
    return "+OK pass accepted"

def handleStat(data, msg):
    return "+OK 1 %i" % msg.size

def handleList(data, msg):
    return "+OK 1 messages (%i octets)\r\n1 %i\r\n." % (msg.size, msg.size)

def handleTop(data, msg):
    cmd, num, lines = data.split()
    assert num == "1", "unknown message number: %s" % num
    lines = int(lines)
    text = + "\r\n\r\n" + "\r\n".join([:lines])
    return "+OK top of message follows\r\n%s\r\n." % text

def handleRetr(data, msg):"message sent")
    return "+OK %i octets\r\n%s\r\n." % (msg.size,

def handleDele(data, msg):
    return "+OK message 1 deleted"

def handleNoop(data, msg):
    return "+OK"

def handleQuit(data, msg):
    return "+OK pypopper POP3 server signing off"

dispatch = dict(

def serve(host, port, filename):
    assert os.path.exists(filename)
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.bind((host, port))
        if host:
            hostname = host
            hostname = "localhost""pypopper POP3 serving '%s' on %s:%s", filename, hostname, port)
        while True:
            conn, addr = sock.accept()
            log.debug('Connected by %s', addr)
                msg = Message(filename)
                conn = ChatterboxConnection(conn)
                conn.sendall("+OK pypopper file-based pop3 server ready")
                while True:
                    data = conn.recvall()
                    command = data.split(None, 1)[0]
                        cmd = dispatch[command]
                    except KeyError:
                        conn.sendall("-ERR unknown command")
                        conn.sendall(cmd(data, msg))
                        if cmd is handleQuit:
                msg = None
    except (SystemExit, KeyboardInterrupt):"pypopper stopped")
    except Exception, ex:
        log.critical("fatal error", exc_info=ex)

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print "USAGE: [<host>:]<port> <path_to_message_file>"
        _, port, filename = sys.argv
        if ":" in port:
            host = port[:port.index(":")]
            port = port[port.index(":") + 1:]
            host = ""
            port = int(port)
        except Exception:
            print "Unknown port:", port
            if os.path.exists(filename):
                serve(host, port, filename)
                print "File not found:", filename
Original code is created by: Daniel Miller

Aug 31, 2015

Install Docker under Ubuntu 14.04 (Trusty)

Docker supports Ubuntu versions:

  • Ubuntu Vivid 15.04 (64-bit)
  • Ubuntu Trusty 14.04 (LTS) (64-bit)
  • Ubuntu Precise 12.04 (LTS) (64-bit)
  • Ubuntu Raring 13.04 and Saucy 13.10 (64 bit) 

For both Vivid and Trusty you need nothing. It will work out of the box. Others will require some modifications. (Updating of some things, like kernel or installing with wget on 13.04)

1. To install docker from a repository do so:
sudo apt-get update
sudo apt-get install
sudo ln -sf /usr/bin/ /usr/local/bin/docker
sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/

2. Now run it with:
sudo apt-get install lxc-docker

3. Make it run on system boot:
sudo update-rc.d defaults

4. Ready to go! Run container with an Ubuntu:
sudo docker run -i -t ubuntu /bin/bash

To disconnect, or detach, from the shell without exiting use the escape sequence Ctrl-p + Ctrl-q

Aug 27, 2015

Remi and EPEL repositories in CentOS

There are 2 common repositories that come nowdays for centos. They contain tasty things, while they are absent in official repositories.

CentOS 5:
sudo rpm -Uvh remi-release-5*.rpm epel-release-5*.rpm

CentOS 6:
sudo rpm -Uvh remi-release-6*.rpm epel-release-6*.rpm

You can check you are successful like so:
ls -1 /etc/yum.repos.d/epel* /etc/yum.repos.d/remi.repo

Now you are only left to activate Remi repository:
sudo vi /etc/yum.repos.d/remi.repo

In [remi] section we need to change enabled=0 into enabled=1. It will look like so:
name=Les RPM de remi pour Enterprise Linux 6 - $basearch

Only need to update now:
yum update -y

Aug 25, 2015

Tmux quick start guide

Tmux is a handy terminal manager that allows you to switch between terminal sessions easily. Without losing history or windows upon ssh disconnects or similar. It is like screen, just better. (First of all because of using client-server based technology... )

Here is my minimal keyboard shortcuts guide that allows you to start using Tmux in a blink of an eye. Endless advanced commands and hotkey combinations you could always find by entering "man tmux" in a terminal.

Tmux is installed quite easily in most of common linux based systems. Just type:
$ sudo apt-get install tmux
$ sudo yum install tmux

This allows you to start using by starting it with
$ tmux a || tmux new
This command first tries to attach to existing running tmux instance and creates new in case it is not found.

Ctrl+b d - Will allow you to disconnect at any time. (This is also a way it is happening when you loose ssh session. How to connect - look earlier)

Each session can have many windows:
Ctrl+b c - Create a new window.
Ctrl+b 0...9 - switch to window #;
Ctrl+b p - switch to previous window;
Ctrl+b n - switch to next window;
Ctrl+b l - switch to next active window (to the one you have switched from to this window);
Ctrl+b & - close this window (Or just type exit in terminal.

One terminal can have multiple windows:
Ctrl+b % - splits current panel in part, vertically;
Ctrl+b " - splits current horizontally;
Ctrl+b →←↑↓ - switch between panels;
Ctrl+b x - close current panel (you could also type exit in a terminal window).

Minus is a scrolling behavior:
Ctrl+b PgUp - enters "copy mode", afterwards:
    PgUp, PgDown - scrolling;
    q - exit copy mode.

Good more complex gist of hotkeys also here:

Mar 5, 2015

Installing MySQL and phpMyAdmin for web development on a Mac OS X 10.9, 10.8, 10.7, 10.6

This is a simple guide on installing a must have SQL database engine for web development. It is often used at the beginning of the journey and I did not find a good guide for it. That made me make myself one and share with others. Hope somebody would benefit from it.

Jan 23, 2015

[Django CMS] Adding plugins inside plugins programatically

I have a task to migrate a website. Old one is plain HTML and new one is Django CMS. I have a script that parses an HTML page from old website in to CMS page and places it into proper place.
Task is to migrate all the page content (that is a CMS TextPlugin) into an interlinked pages setup. Like we have text

<p>blah blah </p><a href="/page/url">Text</a><p> some other text</p>

And I need to change it into CMS LinkPlugin that is nested inside of the TextPlugin. It have become a common standard in the Django CMS world now. Note we need to have a LinkPlugin because of the requirement to interlink pages.
E.g. <a href="/page/url"> is a link to CMS Page object.

The solution is divided into two major parts. First we need to have the link plugin added to a placeholder of the text plugin. It must also be nested by the TextPlugin, like Django CMS admin web UI would do.  So our plugin would look somehow like this in the CMS Page admin:
And the second is modifying the Text Plugin instance itself with placing Link plugin placeholders instead of <a></a> tags we have there.
In this particular example we have a placeholder named "content" and a page containing only one TextPlugin with text in it. We need to modify that text.
First we need to create an instance of a CMSPlugin that is attached to a proper placeholder and is nested in our TextPlugin instance. Creating a Link plugin (an instance of a configuration for that CMS plugin in practice) is a next step to achieve this.
That will be the part 2 of this article. Coming soon.

Jan 22, 2015

Don't put html, head and body tags automatically into beautifulsoup content

Was making a parser recently with BeautifulSoup. Came to the final with rendering contents of the edited text. Like so:
text = "<h1>Test text tag</h1>"
soup = BeautifulSoup(text, "html5")

text = soup.renderContents()
print text
It renders those contents with a result wrapped into the <html>, <head> and <body> tags. So print output looks like so:
'<html><head></head><body><h1>Test text tag</h1></body></html>'
That's a feature of the html5lib library, it fixes HTML that is lacking, such as adding back in missing required elements.

The workaround is simple enough:
text = soup.body.renderContents()
This solution will return an inside of the <html> tag <body>.
Result is:
text = '<h1>Test text tag</h1>'

Dec 20, 2014

Error copying files in Finder of OS X

I have an SD flash card (8 GB Kingston) and a MacBook Pro 13" (Late 2011 model). I also use external card reader time to time.
This problem persisted on all the conditions. The error message was stating:

The Finder can’t complete the operation because some data in “” can’t be read or written.
(Error code -36)

This was happening while copying Photos from my camera (cr2 files). It worked, however, in case of copying files up to 200 MB in total size of batch. It dropped this error message and did stop to copy files upon selecting of lots of RAW files. E.g. all of them and attempting to copy them from a flash drive.

Problem occurred with building miniatures of the CR2 files. Those files are quite heavy photos (25+ MB) and building miniature did take some time. While building those miniatures on both MAC and SD card finder windows it did die.

For me it was enough to change the view from icons to list. E.g.:
This did solve it for me. Other solution (Suspect in case your MAC would be lower processor model and/or less productive hardware version) is to disable building miniatures for files in Finder.

Jul 31, 2014

5 most common Rsync command usage examples

rsync is a command for remote sync.

It is used to synchronise one location to another in a simple way. Location is meant to be local directory, server or remote web server or whatever accessible by ssh.

Advantages of using rsync over other tools is speed and bandwidth requirements. First time rsync copies entire contents of the directory provided and increments changes over next sync times.

Command syntax:

$ rsync options source destination

1. Synchronising local directories:

$ rsync -zvr /tmp/logs/ /tmp/logs1/
building file list ... done
created directory /tmp/logs1

sent 98 bytes  received 48 bytes  292.00 bytes/sec
total size is 0  speedup is 0.00
Used rsync command options here:

  • -z is for compression
  • -v is for verbose output
  • -r is for recursive directory scanning
By default rsync does not preserve timestamps. 

2. Preserve timestamps and permissions during sync:

$ rsync -azvr /tmp/logs/ /tmp/logs1/
building file list ... done

sent 122 bytes  received 48 bytes  340.00 bytes/sec
total size is 0  speedup is 0.00
Option -a preserves symbolic links, timestamps, user permissions and ownership.

3. Synchronize from Local to Remote:

$ rsync -avz /tmp/logs/ root@
root@'s password: 
building file list ... done
created directory /home/user/logs

sent 122 bytes  received 48 bytes  68.00 bytes/sec
total size is 0  speedup is 0.00
It is required to specify username and ip-address of the remote server, while doing synchronization. It is also required to specify the destination directory on the remote server. The format is username@machinename:path. Sometimes depending on your credentials and ssh authentication method you may need to enter a password. Like in this example.

4. Synchronize from Remote to Local:

$ rsync -avz root@ ~/temp/
root@'s password: 
receiving file list ... done

sent 290 bytes  received 11002 bytes  3226.29 bytes/sec
total size is 16956702  speedup is 1501.66
This example is opposite to previous. we have synchronized a list of only changed files here. So speedup is relatively high.
5. View the rsync Progress during Transfer:
$ rsync -avz --progress root@ ~/temp/
root@'s password: 
receiving file list ... 
284 files to consider
        1372 100%    1.31MB/s    0:00:00 (xfer#1, to-check=252/284)
        1024 100% 1000.00kB/s    0:00:00 (xfer#2, to-check=251/284)
        8658 100%    8.26MB/s    0:00:00 (xfer#3, to-check=250/284)
           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=249/284)
        1024 100%  500.00kB/s    0:00:00 (xfer#5, to-check=248/284)
        8632 100%    2.74MB/s    0:00:00 (xfer#6, to-check=247/284)

sent 158 bytes  received 9666 bytes  3929.60 bytes/sec
total size is 16956814  speedup is 1726.06
Running with --progress option showcase detailed progress of server interaction during load operations.

Thats it. Hope you find this article helpful to you. Comments? Suggestions?

Jul 29, 2014

Django adding custom widget to Django Admin

Sometimes you need to get out of standard behaviour of your Django admin. It means expanding its functionality with custom things. In our case we have a model that we want to add a custom html element. Button that must load something via AJAX from another place to be exact.
We have a typical Django polls application for simplicity. Here is its structure:

First of all we need to create a structure. So here comes our model:
from django.db import models

class Poll(models.Model):
    question = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
We have to have our django application admin configuration to display that model:
from django.contrib import admin
from polls.models import Poll
It will look in Django admin like so:
You can see polls available here in admin. You can do all the typical things with Polls model. Add/Delete/Edit... Using standard Django features. Imagine we have something on that polls model that we need to expand. Say, we need a button to visit external site here, made like an additional edit field. Imagine we need to take something on external website. We can post something to that website from our filled django admin form. To do that we need to create a custom form field. I have not found any standard widgets to do that. It is required to write our own field and attach it to the Django admin form.
We do not need own field here. Just a widget. Widgets are standard Django types. It is possible to read wide description of them in the official documentation: Django Widgets. I wont discuss here best ways to implement this task and there might be better ones. However I have chosen a form widget for simplicity.

It is required to have a model expansion form and a custom widget in order to do so. Registration fo that for m in django admin setup is also required. Lets implement this.
Starting with a form and a widget:
from django import forms
from django.utils.safestring import mark_safe
from django.template.loader import render_to_string

from polls.models import Poll

class ButtonWidget(forms.Widget):
    template_name = 'auth_button_widget.html'

    def render(self, name, value, attrs=None):
        context = {
            'url': '/'
        return mark_safe(render_to_string(self.template_name, context))

class PollsForm(forms.ModelForm):
    button = forms.CharField(widget=ButtonWidget)

    class Meta:
        model = Poll
It is located in a newly created file - at the django app - polls directory. Here we have a custom written widget that inherits a typical default Django widget (forms.Widget). we have provided it a template - auth_button_widget.html and a custom render() method that provides extra context for that template.
Next comes the PollsForm class with a custom button field that uses that widget. Note here Meta for that model is specified, indicating we are adding that field to the Polls model add/edit form.
Template, residing in all django templates directory will look like so:
{# auth_button_widget.html #}
<a href="{{ url }}">Go Button</a>
We have our form and widget and a template to render that form element. Time to use that form.

Using this form means modifying our file and registering custom model admin there. It will use our custom form and a widget. Resulting will look like this:
from django.contrib import admin
from polls.models import Poll
from polls.forms import PollsForm

class PollsAdmin(admin.ModelAdmin):
    form = PollsForm, PollsAdmin)
Resulting change will add a custom form to edit or add the Polls model, containing our custom button field. It will look something like so:

Notice the button we have add here. Its just a simple HTML element that needs to be styled and animated by some CSS and JS. It can be done adding Media to our form widget. Resulting new widget will look like this:
# (partially)
class ButtonWidget(forms.Widget):
    template_name = 'auth_button_widget.html'

    class Media:
        js = (
        css = {
            'all': (

    def render(self, name, value, attrs=None):
        context = {
            'url': '/'
        return mark_safe(render_to_string(self.template_name, context))
Its a registration of static files to use in your widget. They will represent a scripting and styling for that field and will be included to the page of django admin form automatically.
For this to function you have to have django static files configured

Content of scripts and styles are out of the scope of this article. Main idea here is to highlight a proper way to add and write a custom widget from the backend perspective.

Comments, suggestions?