Technical articles

Offre de boulot sympa pour hacker junior

Mon, 13 May 2013 03:58:00 +0200

Salut a tous, et particulièrement aux drupalistas et autres artisans du web que ça peut intéresser. Pour ceux qui ne me connaissent pas, voici mon profil sur StackOverflow. Pour ceux qui ne sont pas sur de bien connaitre le mot hacker, ou qui pensent carrément que je fais une erreur en employant ce mot, merci de lire hacker howto par Eric Steven Raymond.

Je cherche quelqu’un de débrouillard en terme de création de sites webs pour me remplacer, avec un minimum d’expérience en drupal - que je ne connais que depuis 2 mois donc je suis facilement remplaçable… Ce job est parfait pour quelqu’un d’un qui n’a pas peur des défis et qui prend son métier très à cœur un peu comme moi.

Il s’agit d’une startup parisienne très sympa, pour laquelle je travailles depuis principalement Valencia (Espagne) oú j’ai encore un appartement.

Bon payeurs, je travaille pour eux depuis Mars 2010. J’ai toujours été payé. Comme c’est bien payé et que je suis un excellent gestionnaire et que je vie dans la campagne en Espagne, il m’est arrivé d’encaisser 4 mois de salaires d’un coup, par choix.

Il y a des possibilités d’évolution. C’est payé entre 1800 et 2800 par mois selon les missions et l’expérience. Les collègues sont des gens exceptionnels avec des petits défauts bien sur comme tout le monde … mais surtout de grandes qualités ! Il suffit d’être un peu ouvert d’esprit.

L’emploi vient avec un Asus Zenbook qui appartient au client, c’est un ultra-portable dernier cri:

8Gb de ram,
256Gb de SSD,
quad core i7,
écran meilleur que HD,
5 heures d’autonomie,
ultra fin, ultra léger, ultra portable.

C’est en télétravail et à plein temps. De plus, il y a des voyages à Paris presque tout frais payés plusieurs fois par an.

Je mets ma carierre en suspens pour des raisons personnelles. Je n’aurais plus du tout le temps de travailler sur internet.

Open Source Windows NT

Tue, 02 Apr 2013 17:28:16 +0200

Seems like ReactOS is rapidely growing.

Drupal 7 with nginx and uwsgi-php example configuration

Wed, 27 Mar 2013 13:30:00 +0100

Drupal is a CMS written in PHP which supports PostgreSQL. It is made for mod_php and Apache, thought it works with uWGSI and Nginx.

When you have tried uWGSI you know why you want this.

Example nginx configuration:

server {
    server_name drupal.example.com;
    root /srv/drupal/www/;

    error_log /tmp/nginx_drupal.log;

    index index.php index.html;

    location / { 
        try_files $uri @rewrite;
    }   

    location @rewrite {
        rewrite ^/(.*)$ /index.php?q=$1;
    }   

    location ~* files/styles {
        access_log off;
        expires 30d;
        try_files $uri @rewrite;
    }   

    location ~ .php$ {
        include uwsgi_params;
        uwsgi_modifier1 14; 
        uwsgi_pass unix:/tmp/uwsgi_drupal.sock;
    }   
}

And an example uwsgi configuration:

[uwsgi]
socket=/tmp/uwsgi_drupal.sock
pidfile=/tmp/uwsgi_drupal.pid
daemonize=/var/log/uwsgi/drupal.log
plugins=php

chdir=/srv/drupal
cheaper=4
close-on-exec=1
harakiri=360
max-requests=128
processes=8
master=1
uid=drupal
gid=drupal
chmod=666
log-5xx=1
vacuum=1
post-buffering=8192

Partition your data with django-parting

Tue, 26 Mar 2013 17:44:16 +0100

Just stumbled on django-parting which provides a nice API to partition your data.

For those not familiar with data partitioning, I recommend reading partitionning on mysql.

PyCon 2013 videos are online !

Thu, 21 Mar 2013 17:46:58 +0100

Check them out.

First 5 Minutes Troubleshooting A Server

Tue, 12 Mar 2013 15:59:26 +0100

A very nice article about troubleshooting a server

PostgreSQL for beginners: Initial configuration

Thu, 07 Mar 2013 05:21:00 +0100

Those are notes taken from the talk “PostgreSQL when it is not your job” by Christophe Pettus from PostgreSQL Experts Inc. at DjangoCon Europe 2012.

This article describes how to make a basic PostgreSQL configuration:

logging,
memory,
checkpoints,
planner.

Note: this article is mostly a transcript from the talk by Christophe Pettus: so send all the cookies to him. Thanks !

That’s around 12 configuration options and you’re done.

Logging

Starting with logging is a good idea because it’s the best way of getting information for configuring the other settings.

be generous with logging; it’s very low-impact on the system,
it’s your best source of information for finding performance problems

Where to log

Three options:

syslog, if you have a syslog infrastructure you like already,
standard format to files - if you’re using tools that need standard format,
otherwise, CSV format to files,

What to log ?

Christophe Pettus recommends this configuration, which you can copy and paste in your postgresql.conf:

log_destination = 'csvlog'
log_directory = 'pg_log
# enable file rotator
logging_collector = on
log_filename = 'postgres-%Y-%m-%d_%H%M%S'
log_rotation_age = 1d
log_retation_size = 1GB
# log statements that take more than 250ms
log_min_duration_statement = 250ms
log_checkpoints = on
log_connections = on
log_disconnections = on
log_lock_waits = on
log_temp_files = 0

Memory configuration

Three things you have to tweak:

shared_buffers,
work_mem,
maintenance_work_mem,

shared_buffers

below 2GB of RAM, set to 20% of total system memory,
below 32GB, set to 25% of total system memory,
avec 32GB, set to 8GB,

work_mem

start low: 32-64MB,
then wait until there is some server load (simulated or real),
look for ‘temporary file’ lines in logs,
it will say “I’m creating temporary file of this size”,
set work_mem to 2-3x the largest temp file you see.

This can be a huge spped-up if set properly ! This is probably the setting that has the most impact on postgresql performance !

But if you find yourself setting this super high like 8GB, you probably don’t want to set it that high. Because every time someone does an operation in postgres like a SORT, it could potentially use that amount of memory per planner.

If you’re having problems of memory exhaustion problems, this is the best setting to back off.

maintenance_work_meme

This is the amount of memory that PostgreSQL uses for indexing or vacuum operations.

10% of system memory, up to 1GB,
Maybe even higher if you are having VACUUM problems,

effective_cache_size

Unlike other parameters we’ve seen, this one won’t actually allocate any memory.

set to the amount of file system cache available,
if you don’t know, set it to 50% of total system memory,

Checkpoints

A checkpoint is a complete flush of dirty buffers to disk.
Potentially a lot of I/O (Input/Output),
Done when the first of two thresholds are hit:
- A particular number of WAL (Write-Ahead-Log) segments have been written … basically you don’t have to worry about that, it’s when a certain amount of database activity has happened.
- A timeout occurs.

Settings part I

# that's the right value according to Chripstophe
wal_buffers = 16MB
checkpoint_completion_target = 0.9
checkpoint_timeout = 10m-30m, # Depends on the restart time

The trade off about checkpoint_timeout is: the higher it goes, you’ll generally get better performance, but the longer it will take postgresql to restart after a crash.

Settings part II

Look for checkpoint entries in the logs
Is it happening more often than checkpoint_timeout ?
Then adjust checkpoint_segments so that checkpoints happen due to timeouts rather than filling segments,

Settings part III, warnings

The WAL can take up to 316MBcheckpoint_segments on disk, so the higher that number goes the more space on disk it will be using … but on modern disks it’s not really that much disk,
Restarting PostgreSQL can take up to checkpoint_timeout (but usually less, like 20% of that),

Planner settings

effective_io_concurrency, set to the number of I/O channels; otherwise ignore it. For example if you have a stack of hard drives or an SSD drive that has 32 channels then set it to the number of channels.
random_page_cost, 3.0 for a typical RAID10 array, 2.0 for a SAN, 1.1 for Amazon EBS, else don’t tweak it.

Foundation 4 is out: a nice Twitter Bootstrap alternative

Fri, 01 Mar 2013 11:48:52 +0100

The new brew of Foundation 4 is out, focusing on mobile first. Worth trying.

dropzone.js: nice jQuery plugin for file uploads

Fri, 22 Feb 2013 13:30:33 +0100

dropzone.js is an open source library that provides drag’n’drop file uploads with image previews

Enable global site packages in an existing virtualenv

Thu, 21 Feb 2013 13:30:40 +0100

Nowadays, virtualenv uses --no-site-packages by default. This means that the created virtualenv will not access global site-packages modules ie. /usr/lib/python2.7/site-packages/.

To enable global site-packages again, just remove this file::

your_env/lib/python2.7/no-global-site-packages.txt

Why would you do that ?

Some modules are complicated to install (xpyb) or take too much time to compile (pyqt). Using the distro packages can be pretty useful !

There's no magic: virtualenv for beginners

Wed, 20 Feb 2013 13:30:29 +0100

Allison from hacker school published an interesting article about virtualenv for beginners.

Django security releases: 1.5rc2, 1.4.4, 1.3.5

Wed, 20 Feb 2013 10:38:15 +0100

Please upgrade.

Intel's 6-year-late copy of PaX's UDEREF (Sep 7 2012)

Tue, 19 Feb 2013 13:30:43 +0100

TL;DR: Intel implements UDEREF equivalent 6 years after PaX, PaX will make use of it on amd64 for improved performance.

Read article ….

Linux 3.8 was released on Mon, 18 Feb 2013.

Tue, 19 Feb 2013 09:38:26 +0100

Linux 3.8 was released !

SourceTree: the git GUI that might work for windows users

Mon, 18 Feb 2013 13:30:32 +0100

SourceTree is a free client for Mercurial and Git. The windows release has been announced.

OpenVim interactive online guide to Vim

Mon, 18 Feb 2013 00:02:50 +0100

Vim is a great text and code editor for command line interface. OpenVim presents an interactive tutorial which seems nice for new beginners.

Jedi-vim/rope takes too long

Fri, 15 Feb 2013 13:30:26 +0100

For those using jedi-vim are indirectly using rope refactoring library.

If you are also using a home-level virtualenv ie. in ~/env/ then your vim instance might take crazy long time.

First things first, rope creates a .ropeproject directory where it thinks the project root is. And it will scan every file in every sub directory. A quick and dirty solution is to create empty .ropeproject folders lower in the $HOME filesystem. Rope will automatically use this folder if it finds no other .ropeproject folder at a lower level.

But, if you want a home-level .ropeproject, then you should configure ~/.ropeproject/config.py. It is well commented, you might end with something like this:

# Custom source folders:  By default rope searches the project
# for finding source folders (folders that should be searched
# for finding modules).  You can add paths to that list.  Note
# that rope guesses project source folders correctly most of the
# time; use this if you have any problems.
# The folders should be relative to project root and use '/' for
# separating folders regardless of the platform rope is running on.
# 'src/my_source_folder' for instance.
prefs['source_folders'] = ['env/src']

# You can extend python path for looking up modules
prefs['python_path'] = '/home/jpic/env/lib/python2.7/site-packages/'

Playlistnow.fm source code is now open !

Fri, 15 Feb 2013 12:59:35 +0100

Playlistnow.fm is a social network based on music made in Python/Django. The catalog is powered by last.fm and the videos come from youtube.

Before its final death, we decided to release the source code under Apache 2.0 license.

Using Ghost.py to test Django applications with LiveServerTestCase

Fri, 15 Feb 2013 00:18:00 +0100

I’m posting this because it took me quite a few hours to get to this point:

Ghost.py seem to have a nicer API and to be faster than selenium so far.

django-smarter: the CRUD app I've been waiting for

Thu, 14 Feb 2013 16:21:41 +0100

django-smarter looks way cool, check it out !