Posts

Running tasks with Celery on Heroku guide

Image
An example project and a basic guide showing how to run Django/Celery on Heroku.
Basic requirements First of all, let's actually set up a typical Django project for this. We would need virtualenvwrapperfor that. One could use any other particular method. I prefer this one.
$ cd dev $ mkvirtualenv dch (dch)$ pip install django (dch)$ django-admin startproject djheroku (dch)$ cd djheroku # Make sure is working:(dch)$ ./manage.py runserver From now I will consider working on a terminal with this (dch) environment on.
Heroku hosting setup We would need our project set up for heroku python server. The docs live HERE, as for moment of this guide writing. One would need to follow and setup a basic heroku project. I will not stop here rewriting official guide as it is good enough.
Installing celery Assuming we have a basic django dyno at heroku here we will continue.
Now let's install Celery and add it to our requirements list (as we had just started, let's just overwrite requirem…

SQLAlchemy (Flask) count model instances by unique values

One comes to a task that has to do with counting items in a database. We will describe the right approach here. Despite being so obvious I did not find much of the docs for junior developers to watch and learn. Here is a sample task and solution:
Let's assume we have a model like so:
classCycle(db.Model):id=db.Column(db.Integer,primary_key=True)object_id=db.Column(db.String,nullable=False) Sample date populated into it will be:
{id:1,object_id:'unique1'},{id:2,object_id:'unique1'},{id:3,object_id:'unique2'},{id:4,object_id:'unique2'},{id:5,object_id:'unique2'},{id:6,object_id:'unique3'} We need to count unique model instances with same object_id. To achieve this relatively simple task one would go straightforward. E.g. Fetch all the Cycle instances with a simple query and then iterate through them via for cycle. Looks like so:
objects=Cycle.query.all()cycles_unique={}forobjectinobjects:ifobject.object_idincycles_unique.items():cycles_un…

PostgreSQL DB with pgAdmin4 access through SSH tunnel

Image
Despite using console most of the time I have a preference to edit PostgreSQL databases through UI. Especially when it comes to remote side. Usually one can access this through $ psql command. However this tends to writing raw SQL queries and a lot of typing in overall.
Here is a way to do it with UI. First one needs to make a tunnel.Command is fairly simple:
ssh -fNg -L 5555:localhost:5432{username}@{host.com} One has a tunnel afterwards. This command opens a SSH connection in the background mapping your local port 5555 to your server’s port 5432 (Postgres’ default port). To understand one can observe the meaning of the flags via $ man ssh to see what each of these flags doing.
 It can be accessed via localhost tools like pgAdmin4 at localhost and port 5555
DB config would look like so:

Run Flask with Debug in PyCharm

Image
I have gotten to setting UP a Flask environment for a project recently. Was completely puzzled by how to run this. I have used to running Django project with python manage.py script. This means pointing out a PyCharm interpreter into a certain point of entry, that is also a .py file also.

In flask way of starting things there is another approach however. Here it is in Flask Docs.
It states to set a variable FLASK_APP and then run a flask run command. This will confuse running a python script way of a project being run. Thus to fix this one needs to set a path to flask binary file. It usually is within a virtual environment binary directory, place where your python interpreter resides.

So to set debugging one needs to set path to script binary one needs:
Script: /path/to/env/bin/flaskScript parameters: runEnvironment variable FLASK_APP=app.pyEnvironment variable FLASK_DEBUG=TrueSet working directory back to your app path (It changes automatically according to script being set) So my wil…

Promise -fication of JS calls.

Image
Found a new pattern to use recently that is called Promise. I really like the way ES6/7 brings new thinking patterns into life nowdays. Here is Promise used instead of old pattern.

It was to give JS method a callback function. This splitting code and making a lot of possibilities for error to come out in this place.

One would write old times according to MDN:
functiongreeting(name){alert('Hello '+name);}functionprocessUserInput(callback){varname=prompt('Please enter your name.');callback(name);}processUserInput(greeting); And now it is made with Promise pattern like so:
letpromise=newPromise((resolve,reject)=>{resolve(prompt('Please enter your name.'));});promise.then((name)=>{alert('Hello '+name);}); In general and briefly this now helps to avoid 'callback hell' with functions passed as arguments and write asynchronous code a sort of in synchronous manner.

This all becomes extra useful upon one having need to load set of data from a differe…

Vagrant error: * Unknown configuration section 'hostmanager'.

Sometimes you get a vagrant environment or boilerplate with a Vagrantfile config in there and do a vagrant up command. And see some errors. like this:
There are errors in the configuration of this machine. Please fix the following errors and try again:Vagrant:* Unknown configuration section 'hostmanager'.
To fix this one needs:
$ vagrant plugin install vagrant-hostmanager Installing the 'vagrant-hostmanager' plugin. This can take a few minutes... Fetching: vagrant-hostmanager-1.8.6.gem (100%) Installed the plugin 'vagrant-hostmanager (1.8.6)'! So command to fix this as follows:
vagrant plugin install vagrant-hostmanager

POP3 Mock (Fake) server using python script

Having a need in POP3 server for my debugging purposes I have used this script.
Letting it to be here in case of anyone would need to do something similar.
Usage is:
$ python pypopper.py 110 email_file.eml

"""pypopper: a file-based pop3 serverUseage:    python pypopper.py <port> <path_to_message_file>"""import logging import os importsocketimport sys import traceback logging.basicConfig(format="%(name)s %(levelname)s - %(message)s")log= logging.getLogger("pypopper") log.setLevel(logging.INFO) class ChatterboxConnection(object): END ="\r\n" def __init__(self, conn): self.conn = conn def __getattr__(self, name):return getattr(self.conn, name) def sendall(self, data, END=END):if len(data)<50: log.debug("send: %r", data)else: log.debug("send: %r...", data[:50]) data += END self.conn.sendall(data) def recvall(self, END=END): …