Jun 27, 2013

Django: How to debug Django

There are several ways of debugging Django. I'll try to cover most common and tell what I know/use time to time. This article may become useful to Django newbies and is not intended to be "the only truth". It may help you know in general about debugging techniques commonly used in Python/Django projects that author is aware of. Let's get started. There are several methods to discuss here.

1. Print out into console.

It may become handy while working with critical bugs that only appear in production for e.g. and are not traceable at development environment... The only way here to run your project manually and do debug in production. Those requirements are so rare that I only use standart (builtin) tools. There is a thing called PDB (Python Debugger). That comes in a standart set of usual python distribution. So it will be available to you at almost any environment where you have console access to your project.
Here is an official documentation for pdb module. And you can use it in many advanced ways. But for the sake of going through I'd just stop on a few usages here.
Assuming this all will be done on a production server. Because this is the most common usecase for this  technique. To debug with pdb you would need to edit your python (Django) script. And add an import statement. e.g.:
import pdb
And then somewhere in code in a place where you need to stop and watch.
pdb.set_trace()
This will basically run your server till the line pdb.set_trace() and stop there bringing you the python console, like the one you would run with executing a command python in your terminal. One main difference you would have all your variables accessed in this point of execution interruption. So you will have ability to type for e.g.:
>>> param
['exapmple', 'values', 'list']
>>> 
assuming this param variable is set before this in code. 
You may also have a list of commands that can lead you to debugging in most ways you can ever need. But most useful is h(elp) command. Anyway that was most of the things you may need to know about pdb and it's usage. Most of the time I set print's and use extremely useful (to my opinion) python command dir. This is how it works:
>>> print param
['exapmple', 'values', 'list']
>>> dir(param)
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
>>> 
print will show a your param value. However dir command acts in another way iterating through object's even hidden values. You may find useful to know what you can do. e.g. it shows methods along with class internal defaults. Anyway it may become useful more often.

Those tools are quite sufficient for server and "raw console" debugging. You may also try to study vim.  Most of the Linux distributions (purport production systems) have this editor already installed by default in distribution. There are many other new and much logically easier editors (From the perspective of Microsoft Word user :) ). But this VIM is a formed "golden standard". And you may require those skills at almost any time. 

Also all of this assumes you have run your server project in some accessible console. So you have ability to read console output. Mostly Django projects are run as a service. So you may need to kill this service and manually run command like python manage.py runserver.
That's almost everything you may need to know about server debugging.

2. High level debugging.

WARNING: Skip this if you are a PyCharm or other IDE user. 
Most of the time debugging is done on your dev station. So it's quite reasonable to use some GUI tools. I prefer to code in IDE. It helps me to debug and it's almost the main reason why I use IDE instead of e.g. TextMate or Sublime (Advanced text editors). So main thing is built in handy graphical debugger. Graphical means you can click your mouse through your logical constructions stored in a heap of your project. Anyway let's move on. My preferable IDE is PyCharm. And it's preferable because of it's simplicity and straightforwardness in working with Django based projects. Especially with those project's virtual environments. You can set up a virtual environment and connect PyCharm afterwards to a ready made project. And it is really simple, in comparison to PyDev in Eclipse, for e.g. I do not mind using Eclipse. But initial set up there might be quite hard and problematic to a junior dev. Anyway it all become kind of a "holly war" comments and I believe they are irrelevant to you, my reader. So let's move on. Assume I'm using PyCharm. 

I have a nice article about setting up both PyCharm and Eclipse to work with a Django project. They are relatively old but may give you a glimpse of how to set up your project with IDE. Farther text assumes you know how to set up a project with a PyCharm IDE. So go have a read about it if not. IN general debugging is a part of the process (python process), that is run by your IDE. So debugger connects to it and interrupts execution of your code where you tell him to. PyCharm has a project run management buttons. They look like this:
TO debug you must know how to set up project with this IDE. It's fairy simple. Note button with a green bug on it. It's our debugger. You must execute your project with this button and selected run configuration instead of green button with play sign :). 
You must see something like this in your debug console. Indicating you have debugger on and connected:
Note the red letters "pydev debugger: " here. It saying that debugger is running. Note tabs Console and Debugger here. Now it is time to set the breakpoints. This is done by clicking on a space near a line number (If you have them on). Set up break point may look something like that:
So now when program comes to this line it wil stop execution showing you the stack of this line. and you can click on it's objects. This is extremely handy tool to understand your project structure and your Django principles, how things work in your and typical Django project in general. So this may look something like this:
Note it is highlighted with blue line. (It may differ according to your theme). But this principle is general to almost all IDE's. Now go down to Debugger tabs mentioned earlier. It contains a request object in my case:
You may assume this can be repeated almost anywhere in your project code. So if some output is wrong you may quickly understand where to search next. Or what to change to fix your bug. It is also a nice exercise for a dev to look through his project's data and how they are modified during code execution. Although you must keep it in mind constantly, sometimes it is quite useful to look at it from a "bird's eye view".  Green play button continues code execution. And buttons on top near the tabs help you to navigate through your code without setting additional breakpoints. Keeping mouse on those buttons steady without a click will bring help with their meaning. So I wont stop here for explanations. They are quite obvious and require experimenting anyway. 

You may even learn data structures in Django this way. e.g. Setting breakpoints on where different data types appear and/or modified by copy-pasted text from a Django tutorial ;). Helps to understand what is made and why by the Django itself, not mentioning your own code. Imagine you can do so in templates, with querysets, requests, model instances, whatever fancy data structures you may come out. Anyway this is basically it with "in IDE" debugging. Let's move on.

3. Browser debugging.

May come in handy upon solving some obvious issues. Common tools for this are your browser's debug console. There are many techniques here too. As there are many browsers and tools too. But I will try to showcase the one's I'm using. So let's mention one more thing. There is a project called django-debug-toolbar. It was quite a standart while I was only starting to learn Django. Unfortunately now it is often missed. I do not need it now, because of IDE debugger and big screen to fit everything. But for a novice it may be handy to have your project's common variables and states in the browser itself. to help avoid typical mistyping or similar "easy" bugs. So it looks like that:
So it's a panel in the right of your page overlaying your page. That has several handy functions and various pluggable features. It is quite handy to view you production data for e.g. You just enable it and it is shown when you need it. You can watch server SQL requests, template context and page load time. Isn't it handy.

But let's really move to the browser debugging. I use Chrome. It has a nice built in debug console. It has improved (IMHO) in a recent year or so. Before this I was using Mozilla firefox with a plugin called Firebug. Both those tools are able to show most you may ever need. E.g. what is in browser of your Django project's page. e.g. request that comes to your Django server and response it outputs. Along with headers, cookies, content and so on. 
Chrome debugger may look like this:
It can be accessed by clicking "Inspect Element" in a right mouse button menu. It will show HTML of a page focusing on this element you have clicked. But it's not all it can do. It has a lot of tabs that I won't be stopping on, because they are relevant to JavaScript debugging, rather then Django's. So you may look at page loads and parts of a HTML, CSS styles and so on. Anyway JavaScript comes with Django project always at now-days. But it is a completely another story. 

Hope this will help you understand how debugging is done in Django projects. Have something to add or either I'm wrong somewhere? Dont hesitate co comment here.

Jun 24, 2013

JavaScript: validate date field in form


You may often require serialization of a form in JS and parsing of it's data. Here is a nice recipe to validate date with browser default methods. I like it because of relative simplicity.

It works on a principle of parsing date's components. We will create a JavaScript Date object from it and check if it is the same as a parsed date string. Date components that are wrong will go out of rage. SO we will have them different from the existing strings. This method is based on that hack.

I use this only to validate the string itself. I am specifying a placeholder of an input field to minimise user possibility to make a mistake. And I usually force user to enter date in my format with this tool. Anyway it's a good idea to put any datepicker component also.

function validate_date(value) {
    /**********************************************
    *    Validating date in format dd/mm/yyyy
    **********************************************/
    var comp = value.split('/');
    var d = parseInt(comp[0], 10);
    var m = parseInt(comp[1], 10);
    var y = parseInt(comp[2], 10);
    var date = new Date(y,m-1,d);
    return (date.getFullYear() == y && date.getMonth() + 1 == m && date.getDate() == d);
}

So I use this with jQuery Plugins like:

  • jQuery.mask() plugin (To specify the exact format of user entered data. In case they are entered by hand. It's useful to just type '01022013' and do not type nasty slashes, entering something like '01/02/2013'.
  • Bootstrap.datepicker(). Because I use bootstrap often. Or there are plenty of native jQuery or jQuery UI plugins for date picking.

Hope this helps someone. Comments?

Jun 21, 2013

Meteor: Run/Bind JavaScript after rendering a template elements

Being studying Meteor myself I'd like to help beginners like me. I have searched for certain decisions that became wired to me after python development. So I'll just leave them here for your consideration.

Imagine a situation when you need to bind your plugins to a newly rendered template. If you are diving into asynchronous JS dev, like I did and have developed AJAX websites... You will, one word, be quite confused because main concepts are different from the one's you used to know. Anyhow I'd try to explain on this simple task.

Assume we have a template:
<template name="site-part">
  <p>{{my-text}}</p>
</template>
And I render it's content like so:
Template.site-part.content = function() {
  return Session.get("my_data");
}
So I have this template piece rendered when "my_data" variable appears in session. And I have events binded to newly rendered webpage:
Template.site-part.events = {
  'click input': function (e,r) {
  // my scripts
  },
  ....
}
So when my page is rendered I'm binding the events to certain DOM elements. This is all good, unless you would need to attach some plugin to your newly rendered DOM elements. E.g. on rendering this <p> element please execute those scripts. In a jQuery world and simple AJAX web pages you would just add a .live() or .on() or even .load() and so on functions that would trigger those your JavaScript code on it's creation in a DOM. It may work in some cases. But Meteor has fansy trigger to bind your code exactly after template is rendered. It is described on a meteor docs page: http://docs.meteor.com/#template_rendered. But I'll make some example here for better understanding:
Template.site-part.rendered = function() {
  $('p').tooltip();
}
This is a .rendered function after main template name. All magic happens there. Line up your jQuery code and bind your events and plugins there.

This .tooltip() is a binding of a simple JavaScript plugin from a bootstrap package. And is here just only for example of of a common task. E.g. animation of elements after rendering. Like adding a tooltip for them after rendering a Meteor template. Hope this all makes sense to you and will help sometime.

Comments? I'm wrong? Suggestions?

Jun 14, 2013

Meteor. Request to host is not allowed by Access-Control-Allow-Origin.

As a novice I'd like to describe a situation when you may need to make calls to a different domain. In my case it was a requirement to access external API. Anyway I'd like to describe here my perspective of usage this in meteor. You would need to get those data and parse them. First thought is to use client. Wrong. You would need to add CORS support to your application.
It means you need to have header Access-Control-Allow-Origin: * added to all of your response objects. This will enable Meteor usage of external domain responses. And requires you to hack Meteor code.

But there is more "proper" way to work with server. Code on a server is executed synchronously and so you can be sure it will be executed and result returned.
So. First we need to make sure we have a Meteor.http package installed. You can install it by executing "meteor add http" in your project root directory.
Meteor.http can work in both synchronous and asynchronous modes. It is differed by defining a callback function to a http.call() method. In our example we will make a http request on external server and work with it on a client.
Here is your client and server code:
url = 'http://.........'

if (Meteor.is_client) {

    // Calling our Meteor server's function 
    // and simply storing data into current session
    Meteor.call('fetchDataFromUrl', function (error, response) {
        Session.set('external_server_data', response)
    });


    // Providing meteor data for template (it renders on data received)
    Template.data.server_data = function () {
        return Session.get('external_server_data');
    };

}

if (Meteor.is_server) {

    Meteor.methods({
        // Declaring a method
        retrieve_doc_types: function () {
            this.unblock();
            return Meteor.http.get(api_url);
        }
    });
    
}
And the template to display their result:
<head>
    <title>data from your server</title>
</head>

<body>
 <h1>Data from your server:</h1>
 {{> data}}
</body>

<template name="data">
 {{server_data}}
</template>
It was quite enough to me to make calls to external API. You can also add something like JSON.parse somewhere in server side to make your data posted in a proper manner. But it is out of the scope of this article. Also please note this.unblock(); method.

Please comment/suggest!