Sep 25, 2012

Django Log Files Viewer documents

Django Log File Viewer.

This is a PYPI package django-log-file-viewer documents.
Github repo: django-log-file-viewer@garmoncheg.github.com

Usage:

Useful to add log files view functionality to your Django admin web site.
Instead of using database log files storage, it gives you ability to store/view log files through GUI.
It requires a directory with Django log files to function. E.g. directory structure:


$ project_dir/logs/:
       applog.log
       applog.log.2012-09-22
       ...
       errors.log
       applog.log.2012-09-22
       ...


Screenshots:


To parse/display these log files you need:

1. Install an app and add it to your settings.py INSTALLED_APPS section:


# settings.py
INSTALLED_APPS = (
    # ...
    'django-log-file-viewer',
    # ...
)

2. Set UP 2 django variables in settings.py:


# settings.py:
LOG_FILES_DIR = '/path/to/your/log/directory'
    # Relative or static path string of your log files directory.
    # I recommend using more pythonic way of defining tis with os module. E.g. :
    # settings.py:
    #
    # LOG_FILES_DIR = os.path.join(APP_PATH, 'testdata', 'log')
    # 
    # where APP_PATH is your app's/project's path.



LOG_FILES_RE = '(?P<date>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\s\[(?P<type>[A-Z]+)\]\s(?P<message>.+)'
    # Is a regex to parse your log file against. It completely depends of your Django logging settings.
    # And table column names (in a parsed logfile) depend from group names you provide in the regexp.
    # E.g. for Django logging server to parse with this regexp you need to have log, as in example
    # django_log_file_viewer/testdata/testing.log file.

    # to produce this log I've added this formatter to my website.
    
    #'formatters': {
    #    'verbose': {
    #        'format': '%(asctime)s [%(levelname)s] %(message)s'
    #    },
    #},

3. And add urls to your main urls section:


# urls.py
urlpatterns = patterns('',

    # Include this before admin to enable app admin url overrides
    # Note url must be the same as admin
    # This is required step
    url(r'^admin/', include('django-log-file-viewer.admin_urls')),
    url(r'^admin/', include(admin.site.urls)),

    # To view with custom views:
    # Optional step
    # Will ad urls like www.example.com/logfiles/
    url(r'', include('django-log-file-viewer.urls')),
)

TODO's:

  • Add pagination to both log files list and log file content

Reference:


Sep 19, 2012

SSH Unix/Lunux Recipes for daily usage.

I've decided to create a post with several recipes for commands I need to use daily. I use them from a Mac OS X Lion default console, except for some mac ports installed... Remote is special linux distribution.

How do I Compress a Whole Linux or UNIX Directory? 

You need to use tar command as follows (syntax of tar command):
tar -zcvf archive-name.tar.gz directory-name
Where,
  • -z: Compress archive using gzip program
  • -c: Create archive
  • -v: Verbose i.e display progress while creating archive
  • -f: Archive File name
For example, you have directory called /home/garmoncheg/data and you would like to compress this directory then you can type tar command as follows:
$ tar -zcvf data.tar.gz /home/garmoncheg/data/
Above command will create an archive file called data.tar.gz in current directory. If you wish to restore your archive then you need to use following command (it will extract all files in current directory):
$ tar -zxvf data.tar.gz
Where,
  • -x: Extract files

Put or Retrieve a file from/to remote directory via SSH

For all those operations you need to use scp command. It is like a cp command under local unix/linux console. Except it does remote transfers.

So to put one file:
scp file.dat garmoncheg.google.com:NewData.txt
copy file "file.dat'' from your current computer/directory to your home directory on garmoncheg.google.com, naming the copy on remote server "NewData.txt''.  
scp file.dat garmoncheg.google.com:
is the same thing, except file will be named like original at local computer HD.
scp file.dat garmoncheg.google.com:/home/garmoncheg/data.txt
with specifying original path to store files at.

To get files you need to reverse paths given via arguments to scp. E.g.:
scp garmoncheg.google.com:data.txt file.txt
This will copy file from your default ssh directory named "data.txt" to local (current directory) file called "file.txt"

To work with directory (get/put files) recursively there is an -r switch:
scp -r project-stage:logfiles /home/garmoncheg/logfiles
This will download directory "logfiles" from server with ssh shortcut "project-stage" into current directory naming it "logfiles"

That's it. There are many more decisions and recipes for those typical enough operations. Dont hesitate to comment with yours ;)