High Fibre Programming

Like most websites, glogster.com periodically sends out an e-newsletter to its user-base.    This is done by a simple PHP script that opens a socket to a mail server and goes through the user database, sending an email to each one.    This approach has worked fine until recently, the number of users has gone up dramatically and now it takes an infeasibly long time to send emails to all our users.

After optimising the script as much as possible, the only alternative option was to run many instances of the script simultaneously.   PHP doesn’t support multithreading, and although PHP does support POSIX style process control ( http://www.php.net/manual/en/book.pcntl.php ) I decided to create a wrapper in C that would handle multiple instances of PHP.   (I haven’t found anything technically wrong with PHP’s process control, I just needed something stable, reliable and potentially able to mulitask other scripts such as Python).

How it works

The wrapper I made (see below for code listing) is fairly simple and works by creating a series of threads, each of which forks and creates a PHP instance (replacing the child process) that runs the emailing script.   The thread from the parent process waits for the PHP process to finish and then ends itself.   Once a thread ends, the main thread creates another one, thereby maintaining a constant number of threads.   Signal handling is handled by a separate thread and can either tell the main thread to stop creating new threads and thereby safely shutdown the program or directly terminate all child processes and thus abruptly end the program.

Almost done

So now I had a way of running any number of PHP processes simultaneously and was almost ready to send emails at lightning speed, however one problem remained.   Once an email has been sent to a user, the database is updated so that I know which user has received which email and thus prevent me from sending the same user the same email more than once.   The problem is; what if one instance of the PHP script reads a user, and before it has time to send them the email and update the database, another PHP instance reads the same user and also sends them the email?   One way to solve this would be through locking the database table before reading and updating each row, however this would slow the database down which is not ideal.

The solution I came up with was to limit the number of simultaneous PHP instances to 8 and have the wrapper send each PHP process a unique instance ID from 0 to 7.   When selecting users from the database, the script looked at the 3 least significant bits (LSBs) of the IDs in the recipients table and selected only those rows whose 3 LSBs equated to the instance ID passed from the wrapper.

SELECT *,
(
	if( (`id` | 1) = `id`, 1, 0) +
	if( (`id` | 2) = `id`, 2, 0) +
	if( (`id` | 4) = `id`, 4, 0)
) AS `IID`
FROM `recipients`
WHERE `sent` = 0
HAVING `IID` = :IID
LIMIT :OFFSET, :BATCH_SIZE;

Just bind the instance ID passed from the wrapper into the query and each instance will now select recipients that are unique to that ID – no two concurrent instances can select the same recipients. Just don’t forget to update each row once each email has been sent.

Finishing touches

We’re almost there, just one more little feature is required and the system will be perfect (sort of). If a thread process has nothing to do then it will return immediately, so we end up with a situation when thread processes are constantly created and destroyed which is not the most efficient use of resources. The solution was to check the exit status of the PHP process. If zero then all is ok, the thread ends and is available to be restarted immediately as normal. If however, the exit status is non-zero then the thread is marked as “sleeping” and cannot be restarted by the main thread for another 30 seconds.

If the PHP script doesn’t find any rows in the database to process then it returns 250 (a non-zero, non-reserved exit status) and so the calling thread in the wrapper sleeps for 30 seconds before trying again to see if there are any new items in the database to process.

This has the added advantage that if anything goes wrong in the PHP script, such as a fatal error, then the thread will sleep and you won’t end up with perpetuate thread cycling.

Code listing

Here is the code for the wrapper. The code should be fairly straightforward, with the above notes and the inline comments you should be able to figure out what’s going on – just don’t forget to compile with the pthread library. Comments are of course very welcome!

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <errno.h>
#include <sys/wait.h>

#define NUM_THREADS 8
#define THREAD_STACK_HEADROOM 1048576

pthread_mutex_t mutexSignal;
int slots[NUM_THREADS];
int handledSignal = -1;

void* signalHandler(void* arg);
void waitForThreads();
void killall();
void *task(void *i);

/**
 * Each thread will do this
 *
 */
void *task(void *i)
{
    sigset_t signalSet;
    int iid;
    char *tid;
    int stat_loc;

    iid = (int)i;

    /* Say hello and show the thread number */
    printf("Thread %d: starting\n", iid);

    /* Spawn a child to run the program. */
    pid_t pid=fork();

    switch (pid)
    {
        case 0:
            setsid();
            sigfillset(&signalSet);
            pthread_sigmask(SIG_UNBLOCK, &signalSet, NULL );

            tid = malloc(11*sizeof(char));

            sprintf(tid, "tid=%d", iid);

            char *argv[]={"php", "./sleep.php", tid, NULL};

            execv("/usr/bin/php",argv);
            free(tid);
            exit(EXIT_FAILURE); /* only if execv fails */
        case -1:
            printf("Thread %d: Fork failed\n", iid);
            slots[iid] = 0;
            break;
        default:
            /*  parent process */
            slots[iid] = pid;

            if (pid == waitpid(pid,&stat_loc,WUNTRACED)) /* wait for child to exit */
            {
                printf("Thread %d: Child finished with exit code: %d", iid, WEXITSTATUS(stat_loc));

                if (WEXITSTATUS(stat_loc) != 0)
                {
                    /* Sleep this thread, set available time to now+30s */
                    printf(" Going to sleep\n");
                    slots[iid] = -1 * (time(0) + 30);
                }
                else
                {
                    /* Free this thread slot */
                    printf(" Returning to pool\n");
                    slots[iid] = 0;
                }
            }
            else
            {
                printf("Thread %d: Bad exit\n", iid);
                slots[iid] = 0;
            }
    }

    printf("Thread %d: Finished\n", iid);

    /* Terminate the thread */
    pthread_exit((void*) i);
}

void killall()
{
    int kv,i;

    for (i=0; i<NUM_THREADS;i++)
    {
        kv = kill(slots[i], SIGINT);
        printf("Killed %d: %s\n", slots[i], kv ==0?"ok":"fail");
    }
}

void waitForThreads()
{
    int i, stoppedThreads;

    printf("Waiting for threads to finish\n");

    while(1)
    {
        stoppedThreads = 0;

        for (i=0; i<NUM_THREADS;i++)
        {
            /* Check if thread has finished or is sleeping */
            if (slots[i] == 0 || slots[i] < -1)
            {
                stoppedThreads++;
            }
        }

        if (stoppedThreads == NUM_THREADS)
        {
            break;
        }
    }
}

void* signalHandler( void* arg )
{
    sigset_t signal_set;
    int sig;

    while (1)
    {
        /* Wait for any and all signals */
        sigfillset(&signal_set);

        sigwait(&signal_set, &sig);

        /* When we get this far, we've caught a signal */

        switch(sig)
        {
            /* SIGQUIT */
            case SIGTERM:
            case SIGQUIT:
                pthread_mutex_lock(&mutexSignal);
                handledSignal = SIGQUIT;
                pthread_mutex_unlock(&mutexSignal);
                break;

            /* SIGINT */
            case SIGINT:
                pthread_mutex_lock(&mutexSignal);
                handledSignal = SIGINT;
                pthread_mutex_unlock(&mutexSignal);
                break;

            /* other signals */
            default:
                pthread_mutex_lock(&mutexSignal);
                handledSignal = 0;
                pthread_mutex_unlock(&mutexSignal);
                break;
        }
    }

    return (void*)0;
}

/**
 * Main
 *
 */
int main()
{
    int rc, i;
    int running = 1;
    size_t stacksize;
    sigset_t signalSet;
    pthread_t threadSignalHandler;
    pthread_t thread[NUM_THREADS];
    pthread_attr_t attr;

    /* Initialise the signal mutex lock */
    pthread_mutex_init(&mutexSignal, NULL);

    /* Initialise and set thread attributes */
    pthread_attr_init(&attr);
    //pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
    pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);

    /* Ensure each thread has a given stack size - portability */
    pthread_attr_getstacksize(&attr, &stacksize);
    printf("Default stack size = %li\n", (long) stacksize);

        /* Determine and set a new stack size */
        stacksize = sizeof(sigset_t) + THREAD_STACK_HEADROOM;
        printf("Amount of stack calculated per thread = %li\n", (long) stacksize);
        pthread_attr_setstacksize (&attr, stacksize);

    /* block all signals */
    sigfillset(&signalSet);
    pthread_sigmask(SIG_BLOCK, &signalSet, NULL );

    /* create the signal handling thread */
    pthread_create(&threadSignalHandler, NULL, signalHandler, NULL);

    while (running > 0)
    {
        /* Find an empty slot */
        for (i=0; i<NUM_THREADS;i++)
        {
            if (slots[i] == 0)  /* Thread is currently not doing anything, so let's give it something to do */
            {
                /* Set this slot as used, -1 is a temporary value before being replaced by the PID of the forked process (prevents race hazards) */
                slots[i] = -1;

                /* This slot is spare */
                printf("Main: creating thread %d\n", i);

                /* Create a new thread */
                rc = pthread_create(&thread[i], &attr, task, (void *)i);

                if (rc)
                {
                    printf("ERROR: return code from pthread_create() is %d\n", rc);
                    exit(1);
                }

                break;
            }
            else if (slots[i] < -1)     /* Thread is sleeping, see if we should wake it up */
            {
                if (time(0) > (-1 * slots[i]) )
                {
                    /* Thread has slept long enough so let's wake it up */
                    slots[i] = 0;
                }
            }
        }

        pthread_mutex_lock(&mutexSignal);

        switch ( handledSignal )
        {
            case -1:
                break;

            case 0:
                /* The case for signals we're not interested in */
                handledSignal = -1;
                break;

            case SIGTERM:
            case SIGQUIT:
                printf("Main: SIGQUIT\n");
                handledSignal = -1;
                running = -1;
                killall();
                break;

            case SIGINT:
                printf("Main: SIGINT\n" );
                handledSignal = -1;
                running = -1;
                break;
        }

        pthread_mutex_unlock(&mutexSignal);

        /* Sleep for a bit - all this thread creation is hard work! */
        usleep(100000);
    }

    waitForThreads();

    printf("Bye\n");
}

And here’s a sample PHP script (but you can of course have the wrapper call anything).

<?php

list($junk, $tid) = explode("=", $argv[1], 2);

printf("PHP: Hello from thread %d\n", $tid);

sleep(mt_rand(4,14));

// Non-zero exit status if the calling thread should sleep
// Chosen 250 because it's not a reserved status
// NOTE: PHP returns 255 on fatal error
exit(250);

Exercises left to the reader

The above code is not really intended as production-ready, (although I have actually used it in production) and there are still plenty of loose ends that could do with tidying up. One thing you might want to do is detach the child PHP processes from the parent process’ output sockets, thereby effectively sending them to the background. For inspiration take a look at this basic daemonising program: A simple daemon in C.

Another nice addition might be to allow the child process to be specified in command line arguments to the wrapper. It would be useful to specify the sleep time for threads and also the maximum number of concurrent threads – this way you wouldn’t have to recompile each time you changed something.

If you find this useful or interesting then comments would be greatly welcomed!

It’s fairly common to need to iterate through all the rows in a given table and perform an action on each. The usual way to do this is fetch rows from the database in batches using LIMIT, specifying an offset and number of rows to be returned.

SELECT * FROM `myBigTable` LIMIT :OFFSET, :ROW_COUNT;

After each batch of rows is processed, the offset is increased by the size of the batch and the process is repeated until all rows have been processed.

Now that’s not exactly rocket science, but there is a problem – as the offset increases, the time taken for the query to execute progressively increases, which can mean processing very large tables will take an extremely long time. The reason is because offset works on the physical position of rows in the table which is not indexed. So to find a row at offset x, the database engine must iterate through all the rows from 0 to x.

The obvious solution would be to use the primary key instead of offset:

SELECT * FROM `myBigTable` WHERE `id` > :OFFSET LIMIT :BATCH_SIZE;

The problem here is that MySQL doesn’t guarantee that rows appear in the table in the same order as the primary key, so you could end up processing some rows twice, and worse still, some rows not at all. Luckily this can be easily solved by using ORDER BY:

SELECT * FROM `myBigTable` WHERE `id` > :OFFSET ORDER BY `id` ASC;

This might seem a bit counterintuitive using ORDER BY when we want to speed things up, but remember we’re ordering on a uniquely indexed column which will be fast anyway, and compared to the alternative of iterating through each row using an offset, it’s a huge improvement.

Conclusion

The general rule of thumb is “never use offset in a limit clause”. For small tables you probably won’t notice any difference, but with tables with over a million rows you’re going to see huge performance increases.

Remember, this doesn’t just apply to iterating through tables but whenever offset is used. For example implementing pagination of large tables, if the first few pages load quickly but subsequent pages load progressively slower then it’s likely this is the cause (or you are missing some indices!)

I just tried to commit my Working Copy when I got the error message:

svn: Entry 'jquery-latest.js' has unexpectedly changed special status

It turns out that someone had created a symlink to whatever is the latest version of jQuery, which apart from being a somewhat dubious solution, should be fine.   However, it seems a Windows user then checked out this file and somehow changed it; either directly or by some strange Windowsy magic.   Anyway, the changed file was then corrupted and the result is the rather cryptic message above.

I tried to remove the offending file:

svn remove jquery-latest.js

But that resulted in:

svn: 'jquery-latest.js' is in the way of the resource actually under version control

Which was simpy resolved by using the --force command:

svn remove --force jquery-latest.js

I recreated the symlink, committed it and it seems now to be ok.

As a follow-up to my previous article on displaying a spinning cursor to display program activity, I have created a more useful example that could be adapted for use.

The original problem was this – I was writing a program that performed a particular task and I needed some way of showing the program was alive and functioning correctly whilst doing the task.

In Windows this is done by changing the mouse cursor into an hourglass, unfortunately on the command line we don’t have such GUI luxuries. One classic option is to create a spinning line by displaying | / – \ and so on.

The example below creates two child threads; one to do a particular job (in this case slowly count to 10) and another thread to display a spinning cursor.

/**
 * Spinning cursor test class
 *
 * @author Nick Giles
 */
public class SpinTest
{
    /**
     * Thread which does the stuff we're interested in
     *
     */
    private Thread stuff;

    /**
     * Thread class which does stuff in the background
     *
     */
    private class Stuff implements Runnable
    {
        public void run()
        {
            int result = 0;

            try
            {
                // Slowly count to 10, although you can of course put whatever
                // you want here, presumably something a bit more useful
                while (result < 10)
                {
                    result++;
                    Thread.sleep(1000);
                }
            }
            catch (InterruptedException e)
            {
                // If interrupted then quietly end the thread, you may well
                // want to handle this in some way
            }
        }
    }

    /**
     * Spinner thread
     *
     */
    private class Spinner implements Runnable
    {
        public void run()
        {
            String[] phases = {"|", "/", "-", "\\"};

            try
            {
                while (true)
                {
                    for (String phase : phases)
                    {
                        System.out.printf("\b"+phase);

                        Thread.sleep(100);
                    }
                }
            }
            catch (InterruptedException e)
            {
                // No need to do anything if interrupted
            }
        }
    }

    /**
     * Handles all shutdown functions
     *
     */
    public class ShutdownHandler extends Thread
    {
        public void run()
        {
            // On interrupt, stop doing stuff
            stuff.interrupt();
        }
    }

    public void doStuff()
    {
        try
        {
            // Attach an object to handle shutdown signals (i.e. ctrl+c)
            Runtime.getRuntime().addShutdownHook(new ShutdownHandler()); 

            // Create a new thread for spinning the cursor
            Thread spinner = new Thread(new Spinner());

            // Create a new thread for doing stuff
            this.stuff = new Thread(new Stuff());

            // Nice message to user
            System.out.printf("Doing stuff...  ");

            // Start doing stuff
            this.stuff.start();

            // Start spinning the cursor
            spinner.start();

            // Check the thread doing stuff is still doing stuff
            while (stuff.isAlive())
            {
                stuff.join(1000);
            }

            System.out.printf("\bDone.\n");

            // The thread doing stuff has finished, so stop spinning
            spinner.interrupt();

            // Wait for the spinning thread to terminate
            spinner.join();

            System.out.println("The End.");
        }
        catch (InterruptedException e)
        {
            System.out.println("Interrupted");
        }
    }

    /**
     * Main method
     *
     */
    public static void main(String[] args)
    {
        SpinTest test = new SpinTest();

        test.doStuff();
    }
}

It’s a fairly basic implementation but it should be fairly simple to extend it for a real-world application.

Update: I have created an updated article which describes a multithreaded approach in Java Spinning command line cursor in Java

I think I must be really bored this morning, I can’t believe I’m actually blogging this, but it might be useful for someone, who knows.   Anyway, I am currently writing a program that sits and does stuff for a very long time, and I need a way to nicely indicate the program is still running and doing its stuff.   So I have created a little method that prints a spinning cursor on the command line.   The implementation would of course have to be multithreaded; one thread to do stuff, the other to spin the cursor – if there’s time I’ll update it to a more complete solution, but for now here’s the spin implementation:

public class Spin
{
    public static void main(String[] args) throws InterruptedException
    {
        String[] phases = {"|", "/", "-", "\\"};

        System.out.printf("Spinning... |");

        while (true)
        {
            for (String phase : phases)
            {
                System.out.printf("\b"+phase);
                Thread.sleep(100);
            }
        }
    }
}

And here it is in PHP.

<?php

    $phases = array("|", "/", "-", "\\");

    printf("Spinning... |");

    while (1)
    {
        foreach ($phases AS $phase)
        {
            printf('%s%s', chr(8), $phase);
            usleep(100000); // Replace this with one iteration of doing stuff
        }
    }

Of course PHP doesn’t support threading so you’d have to call each iteration of the “doing stuff” loop inside the spin loop which would mean you’d get a bit of a jumpy spinning cursor but I think most people could live with that.

I recently installed Eclipse and all of a sudden my Java applications failed to build with the error:

Syntax error, parameterized types are only available if source level is 1.5

After a bit of Googling I figured out that Eclipse installed a different JVM and set it as the default one.   On Ubuntu (at least) the way to change the default JVM is by typing this into the console:

sudo update-alternatives --config java

It will show you which JVM is currently selected and allow you to choose a different one as default.

I just discovered that I need to touch() a file in Java. It appears there isn’t a way to do this using the standard Java library, so I rolled my own:

import java.io.*;
import java.util.Date;

class Touch
{
    /**
     * Touches a given file
     *
     * @author Nick Giles <http://www/4pmp.com/>
     */
    public static void main(String args[])
    {
        try
        {
            // Create a new file object for the file we want to touch
            File f = new File("touch.txt");

            // See if the file already exists
            if (f.exists())
            {
                // The file already exists, so just update its last modified time
                if (!f.setLastModified(System.currentTimeMillis()))
                {
                    throw new IOException("Could not touch file");
                }
            }
            else
            {
                // The file doesn't exist, so create it
                f.createNewFile();
            }
        }
        catch (SecurityException e)
        {
            System.err.println("Security Error: " + e.getMessage());
        }
        catch (IOException e)
        {
            System.err.println("IO Error: " + e.getMessage());
        }
    }
}

Just like the Linux command, if the input is a directory then it will update the last modified time for the directory but not recurse into it. If you need to recurse then the above can easily be extended to recurse itself using f.isDirectory() and f.listFiles(). If anyone needs this then just let me know and I’ll rustle it up, but for now I’ll leave that as en exercise for the reader

A daemon is a process which runs in the background of your computer, periodically carrying out a specific task. The following is an example of a simple daemon written in C. It works by forking to create a child process, the parent then terminates but the child carries on in the background – entering a continuous loop of doing a task and then sleeping. The child process is of course an identical copy of the parent so care must be taken to close all file descriptors, thus detaching the child completely from the calling process. The deamonised process is controlled by sending it signals which it can catch and take action accordingly. In the example below, the process is terminated by sending SIGINT or SIGTERM, but you can of course add in your own handling – for example to re-read config data on SIGHUP.

#include <stdio.h>
#include <signal.h>
#include <syslog.h>
#include <errno.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>

#define DAEMON_NAME "simpledaemon"

    void daemonShutdown();
    void signal_handler(int sig);
    void daemonize(char *rundir, char *pidfile);

    int pidFilehandle;

    void signal_handler(int sig)
    {
        switch(sig)
        {
            case SIGHUP:
                syslog(LOG_WARNING, "Received SIGHUP signal.");
                break;
            case SIGINT:
            case SIGTERM:
                syslog(LOG_INFO, "Daemon exiting");
                daemonShutdown();
                exit(EXIT_SUCCESS);
                break;
            default:
                syslog(LOG_WARNING, "Unhandled signal %s", strsignal(sig));
                break;
        }
    }

    void daemonShutdown()
    {
        close(pidFilehandle);
    }

    void daemonize(char *rundir, char *pidfile)
    {
        int pid, sid, i;
        char str[10];
        struct sigaction newSigAction;
        sigset_t newSigSet;

        /* Check if parent process id is set */
        if (getppid() == 1)
        {
            /* PPID exists, therefore we are already a daemon */
            return;
        }

        /* Set signal mask - signals we want to block */
        sigemptyset(&newSigSet);
        sigaddset(&newSigSet, SIGCHLD);  /* ignore child - i.e. we don't need to wait for it */
        sigaddset(&newSigSet, SIGTSTP);  /* ignore Tty stop signals */
        sigaddset(&newSigSet, SIGTTOU);  /* ignore Tty background writes */
        sigaddset(&newSigSet, SIGTTIN);  /* ignore Tty background reads */
        sigprocmask(SIG_BLOCK, &newSigSet, NULL);   /* Block the above specified signals */

        /* Set up a signal handler */
        newSigAction.sa_handler = signal_handler;
        sigemptyset(&newSigAction.sa_mask);
        newSigAction.sa_flags = 0;

            /* Signals to handle */
            sigaction(SIGHUP, &newSigAction, NULL);     /* catch hangup signal */
            sigaction(SIGTERM, &newSigAction, NULL);    /* catch term signal */
            sigaction(SIGINT, &newSigAction, NULL);     /* catch interrupt signal */

        /* Fork*/
        pid = fork();

        if (pid < 0)
        {
            /* Could not fork */
            exit(EXIT_FAILURE);
        }

        if (pid > 0)
        {
            /* Child created ok, so exit parent process */
            printf("Child process created: %d\n", pid);
            exit(EXIT_SUCCESS);
        }

        /* Child continues */

        umask(027); /* Set file permissions 750 */

        /* Get a new process group */
        sid = setsid();

        if (sid < 0)
        {
            exit(EXIT_FAILURE);
        }

        /* close all descriptors */
        for (i = getdtablesize(); i >= 0; --i)
        {
            close(i);
        }

        /* Route I/O connections */
        close(STDIN_FILENO);
        close(STDOUT_FILENO);
        close(STDERR_FILENO);

        chdir(rundir); /* change running directory */

        /* Ensure only one copy */
        pidFilehandle = open(pidfile, O_RDWR|O_CREAT, 0600);

        if (pidFilehandle == -1 )
        {
            /* Couldn't open lock file */
            syslog(LOG_INFO, "Could not open PID lock file %s, exiting", pidfile);
            exit(EXIT_FAILURE);
        }

        /* Try to lock file */
        if (lockf(pidFilehandle,F_TLOCK,0) == -1)
        {
            /* Couldn't get lock on lock file */
            syslog(LOG_INFO, "Could not lock PID lock file %s, exiting", pidfile);
            exit(EXIT_FAILURE);
        }

        /* Get and format PID */
        sprintf(str,"%d\n",getpid());

        /* write pid to lockfile */
        write(pidFilehandle, str, strlen(str));
    }

    int main()
    {
        /* Debug logging
        setlogmask(LOG_UPTO(LOG_DEBUG));
        openlog(DAEMON_NAME, LOG_CONS, LOG_USER);
        */

        /* Logging */
        setlogmask(LOG_UPTO(LOG_INFO));
        openlog(DAEMON_NAME, LOG_CONS | LOG_PERROR, LOG_USER);

        syslog(LOG_INFO, "Daemon starting up");

        /* Deamonize */
        daemonize("/tmp/", "/tmp/daemon.pid");

        syslog(LOG_INFO, "Daemon running");

        while (1)
        {
            syslog(LOG_INFO, "daemon says hello");

            sleep(1);
        }
    }

The above example is only simple but it serves as a good starting point to create your own daemon. In my next article I will demonstrate how it can be extended to periodically call a PHP script.

If I find enough time I will also demonstrate how it can be extended from being a single-threaded daemon to a multi-threaded daemon, maintaining a continuous pool of x threads, each doing a given task.

I recently upgraded my Ubuntu install to the latest Karmic Koala, and to be honest I’ve not been totally impressed.   A range of little bugs that had been fixed before seem to have reappeared – just little things such as problems with icons in Gnome panel, nothing I couldn’t live with.

However, yesterday I got a message saying that there was less than 2GB left on my hard drive.   The last time I checked I had over 20GB free, so where did it all go?   After a bit of investigating I found two giant 10GB kernel log files /var/log/kern.log.

It seemed that the computer was constantly logging the same messages:

CPU0: Temperature above threshold, cpu clock throttled (total events = 208[ 8973.550089] CPU0: Temperature/speed normal

CPU0: Temperat cpu clock throttled (total events = 2080190)

I had a look on Google and it turns out that it is a bug with the new Karmic Koala update:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/453444

So far there’s no patch for it, so the best solution I came up with was just to turn off the logging.

Step 1:
Look in /etc/rsyslog.conf and it will either have the configuration for system logging, or in the case of my machine it points to include all files in /etc/rsyslog.d/   In my case there was only one file in here so it made things simpler.

Step 2:
Open the configuration file in an editor and find the line which specifies kernel logging and comment it, something like:

#kern.*                         -/var/log/kern.log

Step 3:
Now restart the system log daemon:   sudo restart rsyslog

It is the logging which hogs the processor so by turning it off the problem is at least not noticeable.

The problem

On Glogster we recently made a new little Flash widget to allow users to “skin” their profile pages – allowing them to change the colour of text, backgrounds and so on to make their part of Glogster more personal.

It works by saving all the definitions (i.e. div.p = #ff0000, etc) to the database and then generating a custom CSS file which is loaded on the profile page to override the default style.   We also needed a “preview” mode whereby the changes made in the Flash widget would be immediately visible on the page to allow the user to see what the page would look like before saving the skin.

No problem, I thought, when I first saw the brief.   We can use jQuery to easily change styles dynamically.   Indeed we could, but not pseudo classes for things such as :hover on links.   I did a bit of searching and I couldn’t find a way of doing it using jQuery, so I made one.

The solution

I decided on an object oriented approach as I needed a way to keep track of what style properties the Flash widget has sent and update definitions accordingly.   Using an OO approach allowed me to model components of a stylesheet individually thus allowing client code to construct a new stylesheet by use of these components.   For example if the Flash widget sent a “color” definition for the CSS selector “div#main p”, an then sent another definition, let’s say “font-size” for the same selector, then I would need some way to group these definitions together.   So I made three classes:

StyleSheetElement
Contains a property and value pair for example “color” and “#ff0000″.

StyleSheetTag
Contains StyleSheetElement objects for a given CSS selctor, for example “div#main p”.

StyleSheet
Contains StyleSheetTag objects.   Orders the StyleSheetTag objects and creates CSS output.

The implementation

On to an example:
http://www.4pmp.com/examples/javascript-dynamic-pseudo-classes/test.html

    // Create a new StyleSheet instance
    var sheet = new StyleSheet();

    // Create a CSS element to set colour to green
    var elementBasic = new StyleSheetElement("color", "#00ff00");

    // Attach the CSS element to the CSS description for "a"
    var tagA = sheet.addElementToTag("a", elementBasic);

        // Set the order of the tag for "a"
        tagA.order = 2;

    // Create a CSS element to set colour to red
    var elementHover = new StyleSheetElement("color", "#ff0000");

    // Attach the CSS element to the CSS description for "a:hover"
    var tagHover = sheet.addElementToTag("a:hover", elementHover);

        // Set the order of the tag for "a:hover", will appear beore "a"
        tagHover.order = 1;

    // Create a stylesheet for the new elements and add it to the page (replaces it if one already exists)
    addInlineStyleSheet(sheet);

The first step is to create a SyleSheet object that will contain the style definitions and create the CSS.

The second step is to create a StyleSheetElement for the property you want to set and the value you want to set it to.

Next, you need to add that element to the StyleSheet object along with a CSS selector.   If the StyleSheet does not already contain a tag for the given CSS descriptor then it will first of all create one, then it will see if the tag already contains an element for the same property as the element being added.   If it does then the new element replaces the old one, otherwise the new element is just added to the tag.

You may notice that StyleSheet::addElementToTag() returns the tag that the element was added to.   This is useful if you want to set the order of the tags.   Sometimes you will have tags that need to appear higher up in the resulting CSS, set the StyleSheetTag.order property to achieve this, the smaller the number the nearer the top it will appear.

The final stage is to generate the CSS and add/replace it in the page.   To do this, just call addInlineStyleSheet() and pass it the StyleSheet object you’ve just made.

What about Internet Explorer?

Of course, IE managed to throw a spanner into the works by not supporting changing inline stylesheets.   Well, it does support them in that it’s possible to change the actual HTML, but it has no effect on the rendering of the page.   Great.   So I found a workaround, if the browser is a flavour of IE, then the addInlineStyleSheet() function compiles the StyleSheet object not to CSS but to a list of tags and CSS rules which are then applied in turn by calling document.styleSheets.inlinestyle.addRule().

Download

StyleSheet.001.js