Spinning command line cursor in Java and PHP

Update: I have created an updated article which describes a multithreaded approach in Java Spinning command line cursor in Java

I think I must be really bored this morning, I can’t believe I’m actually blogging this, but it might be useful for someone, who knows.   Anyway, I am currently writing a program that sits and does stuff for a very long time, and I need a way to nicely indicate the program is still running and doing its stuff.   So I have created a little method that prints a spinning cursor on the command line.   The implementation would of course have to be multithreaded; one thread to do stuff, the other to spin the cursor – if there’s time I’ll update it to a more complete solution, but for now here’s the spin implementation:


public class Spin
{
    public static void main(String[] args) throws InterruptedException
    {
        String[] phases = {"|", "/", "-", "\\"};
        
        System.out.printf("Spinning... |");
        
        while (true)
        {
            for (String phase : phases)
            {
                System.out.printf("\b"+phase);
                Thread.sleep(100);
            }
        }
    }
}

And here it is in PHP.


Of course PHP doesn’t support threading so you’d have to call each iteration of the “doing stuff” loop inside the spin loop which would mean you’d get a bit of a jumpy spinning cursor but I think most people could live with that.

Posted in Java, PHP | Tagged , , , , , | 2 Comments

Syntax error, parameterized types are only available if source level is 1.5

I recently installed Eclipse and all of a sudden my Java applications failed to build with the error:

Syntax error, parameterized types are only available if source level is 1.5

After a bit of Googling I figured out that Eclipse installed a different JVM and set it as the default one.   On Ubuntu (at least) the way to change the default JVM is by typing this into the console:

sudo update-alternatives --config java

It will show you which JVM is currently selected and allow you to choose a different one as default.

Posted in Java | Tagged , , , , , , | 5 Comments

Java: touch – set file last modified time

I just discovered that I need to touch() a file in Java. It appears there isn’t a way to do this using the standard Java library, so I rolled my own:

import java.io.*;
import java.util.Date;

class Touch
{
    /**
     * Touches a given file
     *
     * @author Nick Giles 
     */
    public static void main(String args[])
    {
        try
        {
            // Create a new file object for the file we want to touch
            File f = new File("touch.txt");

            // See if the file already exists
            if (f.exists())
            {
                // The file already exists, so just update its last modified time
                if (!f.setLastModified(System.currentTimeMillis()))
                {
                    throw new IOException("Could not touch file");
                }
            }
            else
            {
                // The file doesn't exist, so create it
                f.createNewFile();
            }
        }
        catch (SecurityException e)
        {
            System.err.println("Security Error: " + e.getMessage());
        }
        catch (IOException e)
        {
            System.err.println("IO Error: " + e.getMessage());
        }
    }
}

Just like the Linux command, if the input is a directory then it will update the last modified time for the directory but not recurse into it. If you need to recurse then the above can easily be extended to recurse itself using f.isDirectory() and f.listFiles(). If anyone needs this then just let me know and I’ll rustle it up, but for now I’ll leave that as en exercise for the reader :-)

Posted in Java | Tagged , , , , | Leave a comment

A Simple Daemon in C

(Before diving into creating your own daemon, you might want to take a look at the Fat Controller which is a program which can daemonise anything. If you need to daemonise something, run something repeatedly or parallelise something, then it may well be worth taking a look.)

A daemon is a process which runs in the background of your computer, periodically carrying out a specific task. The following is an example of a simple daemon written in C. It works by forking to create a child process, the parent then terminates but the child carries on in the background – entering a continuous loop of doing a task and then sleeping. The child process is of course an identical copy of the parent so care must be taken to close all file descriptors, thus detaching the child completely from the calling process. The deamonised process is controlled by sending it signals which it can catch and take action accordingly. In the example below, the process is terminated by sending SIGINT or SIGTERM, but you can of course add in your own handling – for example to re-read config data on SIGHUP.

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#define DAEMON_NAME "simpledaemon"


    void daemonShutdown();
    void signal_handler(int sig);
    void daemonize(char *rundir, char *pidfile);


    int pidFilehandle;


    void signal_handler(int sig)
    {
        switch(sig)
        {
            case SIGHUP:
                syslog(LOG_WARNING, "Received SIGHUP signal.");
                break;
            case SIGINT:
            case SIGTERM:
                syslog(LOG_INFO, "Daemon exiting");
                daemonShutdown();
                exit(EXIT_SUCCESS);
                break;
            default:
                syslog(LOG_WARNING, "Unhandled signal %s", strsignal(sig));
                break;
        }
    }
    
    void daemonShutdown()
    {
        close(pidFilehandle);
    }

    void daemonize(char *rundir, char *pidfile)
    {
        int pid, sid, i;
        char str[10];
        struct sigaction newSigAction;
        sigset_t newSigSet;
        
        /* Check if parent process id is set */
        if (getppid() == 1)
        {
            /* PPID exists, therefore we are already a daemon */
            return;
        }

        /* Set signal mask - signals we want to block */
        sigemptyset(&newSigSet);
        sigaddset(&newSigSet, SIGCHLD);  /* ignore child - i.e. we don't need to wait for it */
        sigaddset(&newSigSet, SIGTSTP);  /* ignore Tty stop signals */
        sigaddset(&newSigSet, SIGTTOU);  /* ignore Tty background writes */
        sigaddset(&newSigSet, SIGTTIN);  /* ignore Tty background reads */
        sigprocmask(SIG_BLOCK, &newSigSet, NULL);   /* Block the above specified signals */
        
        /* Set up a signal handler */
        newSigAction.sa_handler = signal_handler;
        sigemptyset(&newSigAction.sa_mask);
        newSigAction.sa_flags = 0;
        
            /* Signals to handle */
            sigaction(SIGHUP, &newSigAction, NULL);     /* catch hangup signal */
            sigaction(SIGTERM, &newSigAction, NULL);    /* catch term signal */
            sigaction(SIGINT, &newSigAction, NULL);     /* catch interrupt signal */

        
        /* Fork*/
        pid = fork();
        
        if (pid < 0)
        {
            /* Could not fork */
            exit(EXIT_FAILURE);
        }
        
        if (pid > 0)
        {
            /* Child created ok, so exit parent process */
            printf("Child process created: %d\n", pid);
            exit(EXIT_SUCCESS);
        }
        
        /* Child continues */
        
        umask(027); /* Set file permissions 750 */
        
        /* Get a new process group */
        sid = setsid();
        
        if (sid < 0)
        {
            exit(EXIT_FAILURE);
        }
        
        /* close all descriptors */
        for (i = getdtablesize(); i >= 0; --i)
        {
            close(i);
        }
        
        /* Route I/O connections */

        /* Open STDIN */
        i = open("/dev/null", O_RDWR);

        /* STDOUT */
        dup(i);

        /* STDERR */
        dup(i);
        
        chdir(rundir); /* change running directory */
        
        /* Ensure only one copy */
        pidFilehandle = open(pidfile, O_RDWR|O_CREAT, 0600);
        
        if (pidFilehandle == -1 )
        {
            /* Couldn't open lock file */
            syslog(LOG_INFO, "Could not open PID lock file %s, exiting", pidfile);
            exit(EXIT_FAILURE);
        }
        
        /* Try to lock file */
        if (lockf(pidFilehandle,F_TLOCK,0) == -1)
        {
            /* Couldn't get lock on lock file */
            syslog(LOG_INFO, "Could not lock PID lock file %s, exiting", pidfile);
            exit(EXIT_FAILURE);
        }

        
        /* Get and format PID */
        sprintf(str,"%d\n",getpid());
        
        /* write pid to lockfile */
        write(pidFilehandle, str, strlen(str));
    }
    
    int main()
    {
        /* Debug logging
        setlogmask(LOG_UPTO(LOG_DEBUG));
        openlog(DAEMON_NAME, LOG_CONS, LOG_USER);
        */
        
        /* Logging */
        setlogmask(LOG_UPTO(LOG_INFO));
        openlog(DAEMON_NAME, LOG_CONS | LOG_PERROR, LOG_USER);
        
        syslog(LOG_INFO, "Daemon starting up");

        /* Deamonize */
        daemonize("/tmp/", "/tmp/daemon.pid");
        
        syslog(LOG_INFO, "Daemon running");
        
        while (1)
        {
            syslog(LOG_INFO, "daemon says hello");
            
            sleep(1);
        }
    }

The above example is only simple but it serves as a good starting point to create your own daemon. In my next article I will demonstrate how it can be extended to periodically call a PHP script.

If I find enough time I will also demonstrate how it can be extended from being a single-threaded daemon to a multi-threaded daemon, maintaining a continuous pool of x threads, each doing a given task.

Posted in C | Tagged , , , , , , , , | 11 Comments

Ubuntu 9.10 Karmic Koala: Temperature above threshold, cpu clock throttled

I recently upgraded my Ubuntu install to the latest Karmic Koala, and to be honest I’ve not been totally impressed.   A range of little bugs that had been fixed before seem to have reappeared – just little things such as problems with icons in Gnome panel, nothing I couldn’t live with.

However, yesterday I got a message saying that there was less than 2GB left on my hard drive.   The last time I checked I had over 20GB free, so where did it all go?   After a bit of investigating I found two giant 10GB kernel log files /var/log/kern.log.

It seemed that the computer was constantly logging the same messages:

CPU0: Temperature above threshold, cpu clock throttled (total events = 208[ 8973.550089] CPU0: Temperature/speed normal

CPU0: Temperat cpu clock throttled (total events = 2080190)

I had a look on Google and it turns out that it is a bug with the new Karmic Koala update:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/453444

So far there’s no patch for it, so the best solution I came up with was just to turn off the logging.

Step 1:
Look in /etc/rsyslog.conf and it will either have the configuration for system logging, or in the case of my machine it points to include all files in /etc/rsyslog.d/   In my case there was only one file in here so it made things simpler.

Step 2:
Open the configuration file in an editor and find the line which specifies kernel logging and comment it, something like:

#kern.*                         -/var/log/kern.log

Step 3:
Now restart the system log daemon:   sudo restart rsyslog

It is the logging which hogs the processor so by turning it off the problem is at least not noticeable.

Posted in Linux | Tagged , , , , , , , , , , , , , , | 7 Comments

Dynamic CSS pseudo class styles with jQuery

The problem

On Glogster we recently made a new little Flash widget to allow users to “skin” their profile pages – allowing them to change the colour of text, backgrounds and so on to make their part of Glogster more personal.

It works by saving all the definitions (i.e. div.p = #ff0000, etc) to the database and then generating a custom CSS file which is loaded on the profile page to override the default style.   We also needed a “preview” mode whereby the changes made in the Flash widget would be immediately visible on the page to allow the user to see what the page would look like before saving the skin.

No problem, I thought, when I first saw the brief.   We can use jQuery to easily change styles dynamically.   Indeed we could, but not pseudo classes for things such as :hover on links.   I did a bit of searching and I couldn’t find a way of doing it using jQuery, so I made one.

The solution

I decided on an object oriented approach as I needed a way to keep track of what style properties the Flash widget has sent and update definitions accordingly.   Using an OO approach allowed me to model components of a stylesheet individually thus allowing client code to construct a new stylesheet by use of these components.   For example if the Flash widget sent a “color” definition for the CSS selector “div#main p”, an then sent another definition, let’s say “font-size” for the same selector, then I would need some way to group these definitions together.   So I made three classes:

StyleSheetElement
Contains a property and value pair for example “color” and “#ff0000”.

StyleSheetTag
Contains StyleSheetElement objects for a given CSS selctor, for example “div#main p”.

StyleSheet
Contains StyleSheetTag objects.   Orders the StyleSheetTag objects and creates CSS output.

The implementation

On to an example:
http://www.4pmp.com/examples/javascript-dynamic-pseudo-classes/test.html

    // Create a new StyleSheet instance
    var sheet = new StyleSheet();

    // Create a CSS element to set colour to green
    var elementBasic = new StyleSheetElement("color", "#00ff00");

    // Attach the CSS element to the CSS description for "a"
    var tagA = sheet.addElementToTag("a", elementBasic);

        // Set the order of the tag for "a"
        tagA.order = 2;

    // Create a CSS element to set colour to red
    var elementHover = new StyleSheetElement("color", "#ff0000");

    // Attach the CSS element to the CSS description for "a:hover"
    var tagHover = sheet.addElementToTag("a:hover", elementHover);

        // Set the order of the tag for "a:hover", will appear beore "a"
        tagHover.order = 1;

    // Create a stylesheet for the new elements and add it to the page (replaces it if one already exists)
    addInlineStyleSheet(sheet);

The first step is to create a SyleSheet object that will contain the style definitions and create the CSS.

The second step is to create a StyleSheetElement for the property you want to set and the value you want to set it to.

Next, you need to add that element to the StyleSheet object along with a CSS selector.   If the StyleSheet does not already contain a tag for the given CSS descriptor then it will first of all create one, then it will see if the tag already contains an element for the same property as the element being added.   If it does then the new element replaces the old one, otherwise the new element is just added to the tag.

You may notice that StyleSheet::addElementToTag() returns the tag that the element was added to.   This is useful if you want to set the order of the tags.   Sometimes you will have tags that need to appear higher up in the resulting CSS, set the StyleSheetTag.order property to achieve this, the smaller the number the nearer the top it will appear.

The final stage is to generate the CSS and add/replace it in the page.   To do this, just call addInlineStyleSheet() and pass it the StyleSheet object you’ve just made.

What about Internet Explorer?

Of course, IE managed to throw a spanner into the works by not supporting changing inline stylesheets.   Well, it does support them in that it’s possible to change the actual HTML, but it has no effect on the rendering of the page.   Great.   So I found a workaround, if the browser is a flavour of IE, then the addInlineStyleSheet() function compiles the StyleSheet object not to CSS but to a list of tags and CSS rules which are then applied in turn by calling document.styleSheets.inlinestyle.addRule().

Download

StyleSheet.001.js

Shout me a coffee

If this article has been useful to you and you want to say “thanks”, then why not shout me a coffee by donating through PayPal?





Thanks!

Posted in Javascript | Tagged , , , , , , , , , , , | 6 Comments

Linked Lists in C: push, pop, shift and unshift

A sound understanding of linked lists and how to use them is a prerequisite for programming in C, or indeed most programming languages for that matter.   Most scripting languages provide built-in helper functions for dealing with arrays and linked lists, C on the other hand doesn’t – so you have to do it yourself.   Luckily though, there’s really not that much to it, but if you’re new to C then it might be quite daunting to realise you’re in the deep end with no arm bands!

There are plenty of great books and online tutorials that cover the topic very well, however personally I prefer to look at working examples and so I have created a few for those who are already aware of the theory and want to get on with programming.

All of the examples in this tutorial use a fairly simple struct for linked list nodes containing just a pointer to an integer and the next node in the list.   (You can of course change your nodes to better suit your requirements, but you will have to change the helper functions a bit to reflect this.)

It should be fairly simple to figure out what what’s going on from the inline comments, so I’ll stop waffling now and get on with the code!

Node

Here is the struct that defines each item in the list.

struct node
{
    int val;
    struct node *next;
};

Note that I have decided to store each item’s data as an integer, but you can of course choose other types so long as you alter the functions below to accommodate this type. Basically, the struct can be whatever so long as it has a pointer to the next item in the list.

Push

Creates a new node and add it onto the end of the list.

/**
 * Adds a node onto the end of the list
 *
 */
void push(struct node** item, int data)
{
    struct node *current;
    struct node *endItem;
    
    current = *item;
    
    /* See if the passed value is a node */
    if (current)
    {
        /* See if this is the last node in the list */
        if (current->next == NULL)
        {
            /* This is the last item, so create a new node to go on the end */
            
            /* Allocate memory for the new node */
            endItem = malloc(sizeof(struct node));
            
            /* Set the value */
            endItem->val = data;
            
            /* Set the pointer to the next item to be NULL (there isn't a next, this will be the last item) */
            endItem->next  = NULL;
            
            /* Set the next pointer in the last item to the new item, making the new item now the last one */
            current->next = endItem;
            
            return;
        }
        else
        {
            /* This is not the last item, recurse to the next one */
            return push(¤t->next, data);
        }
    }
    
    /* There are no nodes in the list, so create a new node and it will be the first in the list */
    endItem = malloc(sizeof(struct node));
    
    /* Set the value */
    endItem->val = data;
    
    /* Set the pointer to the next item to be NULL (there isn't a next, this will be the last item) */
    endItem->next  = NULL;
    
    /* Set the passed pointer to point to the new node */
    *item = endItem;
}

Pop

Removes the last node from the end of the list and return its value.

/**
 * Remove the last item and return the value
 *
 */
int pop (struct node **item, int *p)
{
    struct node *current;
    
    current = *item;
    
    if (current)
    {
        /* See if the current item specifies a proceding item */
        if (current->next)
        {
            /* There is a next item, so see if it is the last one */
            if (current->next->next == NULL)
            {
                /* The next item is the last one, so get its value */
                *p = current->next->val;
                
                /* Free the memory reserved for the last item */
                free(current->next);
                
                /* Remove the pointer to the last item */
                current->next = NULL;
                
                return 0;
            }
            
            /* Move on to the next item */
            return pop(¤t->next, p);
        }
        
        /* There aren't any more items, this is the last one and therefore also the first */
        *p = current->val;
        
        /* Release the memory reserved for it */
        free(*item);
        
        /* Set the head node to NULL */
        *item = NULL;
        
        return 0;
    }
    
    /* There are no items in the list */
    p = NULL;
    
    return -1;
}

Shift

Removes the first element from the list and return its value.

/**
 * Remove the first item in the list
 *
 */
int shift(struct node **head, int *p)
{
    struct node *firstNode;
  
    firstNode = *head;

    /* See if there is a first node */
    if (firstNode == NULL)
    {
        /* There isn't, so return -1*/
        return -1;
    }
     
    /* Set the passed pointer, the pointer to the first node in the list, to the next node */
    *head = firstNode->next;
    
    /* Get the value from the first node before we free() it */
    *p = firstNode->val;
    
    /* Deallocate the first node */
    free(firstNode);
    
    /* Return 0, all ok */
    return 0;
}

Unshift

Creates a new node and add it onto the beginning of the list.

/**
 * Add a node onto the beginning of the list
 *
 */
void unshift(struct node** head, int data)
{
    /* Declare and reserve memory for the new item */
    struct node* newNode = malloc(sizeof(struct node));
    
    /* Se the new node's value */
    newNode->val = data;
    
    /* Set the next item to be the item that was previously first */
    newNode->next = *head;
    
    /* Set the pointer to the first item to this new item */
    *head = newNode;
}
Things to bear in mind
pop() and shift(), there is a call to free() on the node we are removing.   Free() does exactly what it says on the tin; it frees the memory reserved for the variable addressed by the the pointer it is passed.   Note that before we create a new node, we must first allocate memory for it by calling malloc().   Free() deallocates this space so that the process can re-use it again in the future.   If we didn’t call free() then we would have to keep using more and memory to store new nodes in, which could soon get out of hand if you create and delete lots of nodes.

So how do we make sure we’re not eating memory?

I made a simple function (see below) that continuously adds and then removes nodes onto and from the list – if we are correctly freeing memory then we should see that the memory used by the program when run will be stable.

int main()
{
    struct node *head;
    int i, j;

    head = NULL;
    
    while (1)
    {
    
        for (i=1;i<6;i++)
        {
            /* Add to the beginning */
            unshift(&head, i);
            printf("Unshift: %d\n", i);
        }
        
        for (i=6;i<11;i++)
        {
            /* Add to the end */
            push(&head, i);
            printf("Push: %d\n", i);
        }
        
        for (i=1;i<6;i++)
        {
            /* Remove from the end */
            if (pop(&head, &j) == 0)
            {
                printf("Pop: %d\n", j);
            }
        }
        
        for (i=1;i<6;i++)
        {
            /* Remove from the beginning */
            if (shift(&head, &j) == 0)
            {
                printf("Shift: %d\n", j);
            }
        }
        
        /* Functions in a different order */
        
        for (i=1;i<6;i++)
        {
            /* Add to the end */
            push(&head, i);
            printf("Push: %d\n", i);
        }

        for (i=6;i<11;i++)
        {
            /* Add to the beginning */
            unshift(&head, i);
            printf("Unshift: %d\n", i);
        }

        
        for (i=1;i<6;i++)
        {
            /* Remove from the beginning */
            if (shift(&head, &j) == 0)
            {
                printf("Shift: %d\n", j);
            }
        }

        for (i=1;i<6;i++)
        {
            /* Remove from the end */
            if (pop(&head, &j) == 0)
            {
                printf("Pop: %d\n", j);
            }
        }
    }
    
    return 0;    
}

Compile:
gcc llist.c -ollist

Don’t forget rights
chmod +x llist

Run:
./llist

Now, in another terminal we can watch how much memory the program is running:
watch 'ps aux | grep llist'

Just for fun
Comment one of the free() calls from either pop() or shift(), recompile, run and watch the new program.   You should see the memory it is using steadily inrease.

Download:

Here’s a file containing all of the above functions and the test.
Linked List Tutorial v1.0

Posted in C | Tagged , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Transparent aggregation in PHP5

The other day a Ruby developer was extolling the virtues of Ruby on Rails to me, in particular the use of “acts_as_” to make a class magically include all the behaviour from another. It isn’t inheritance, the relationship isn’t “is a” but rather “has a” – it’s a sort of transparent aggregation with all the methods and properties of the aggregated object magically available through the containing object. At least that’s what it looks like, from my somewhat limited RoR experience.

Personally, I would rather achieve this functionality by using the Strategy Pattern – it requires more boiler-plate code but I think the end result is much clearer. If you’re reading this via o2 mobile broadband or another portable connection to the web, then prepare to miss your stop on the train, as my temptations to re-work the functionality issues began to take over just lower down the page.

Anyway, not to be outdone, I set about implementing RoR’s acts_as in PHP.   My implementation works by using a class called Aggregatable which allows all child classes to aggregate, or “act_as”, other standard classes.   I’ll start with an example with two simple classes; Binman and Postman which each do different jobs.

Here are the classes:

class Binman
{
    public function collectRubbish()
    {
        printf("Collecting rubbish\n");
    }
}
class Postman
{
    public function deliverLetter()
    {
        printf("Delivering a letter\n");
    }
}

We can use the classes like this:

$binman = new Binman();
$binman->collectRubbish();    // Prints "Collecting rubbish"

$postman = new Postman();
$postman->deliverLetter();    // Prints "Delivering a letter"

So far so good, but what if I wanted a CasualWorker class to be able to both collect rubbish and deliver letters?   I could of course copy the methods from the Binman class and the Postman class into the CasualWorker class, but this would be rather inflexible – suppose if we needed to change the Binman behaviour we would have to update both the Binman and CasualWorker classes.   Instead we make the CasualWorker class a child of the Aggregatable class and aggregate the Binman and Postman classes.   That might sound a bit complex so I’ll illustrate with an example.

Here’s the CasualWorker class:

class CasualWorker extends Aggregatable
{
    public function __construct()
    {
        $this->aggregate("Binman");
        $this->aggregate("Postman");
    }
}

The CasualWorker class can now do everything that the Binman and Postman classes can:

$casualWorker = new CasualWorker();
$casualWorker->collectRubbish();    // Prints "Collecting rubbish"
$casualWorker->deliverLetter();    // Prints "Delivering a letter"

Great!   It works!   But how?

The Aggregatable class contains an array of instances of all the classes it has been asked to aggregate.   It also makes use of PHP’s magic method, __call(), so that when client code invokes a method that is not defined by it or the child class it is extending, it loops through the objects it is aggregating to see if any of those define the method.   If an object is found that defines the requested method then it is run and the result returned – just as if the invoked class defined the method.

class Aggregatable
{
    /**
     * Store of aggregated objects
     *
     */
    private $aggregated = array();

    /**
     * Aggregates objects
     *
     * @param string Classname
     */
    protected function aggregate($class)
    {
        // Check an instance of this class has not already been aggregated
        if (array_key_exists($class, $this->aggregated))
        {
            throw new Exception(sprintf("Class already aggregated: %s", $class));
        }

        // Add a new instance of the class to the store
        $this->aggregated[$class] = new $class();
    }

    /**
     * Magic method - catch calls to undefined methods
     *
     * @param String method name
     * @param array Arguments
     */
    public function __call($method, $arguments)
    {
        // Loop through the aggregated objects
        foreach ($this->aggregated as $subject)
        {
            // Check each object to see if it defines the method
            if (method_exists($subject, (string) $method))
            {
                // Object defines the requested method, so call it and return the result
                return call_user_func_array(array($subject, (string) $method), $arguments);
            }
        }

        throw new Exception(sprintf("Method not found: %s", $method));
    }
}

What about properties?

I’ve kept the example above fairly simple but there’s no reason why it couldn’t be extended to cover properties using the other Magic Functions __set(), __get() and __unset().   In fact, in the example in the download (see below) I’ve implemented basic support for properties.

Conclusion

Like I’ve said before, I’m not a big fan of this sort of “magic”.   It seems to me to break the golden rule of object oriented programmed, as mentioned in the Gang Of Four book; “program to an interface, not an implementation”.   The interface the class provides is defined by the public properties and methods available, or even better, from the Interfaces it implements.   By adding this sort of run-time dynamism, the client code can never be sure what the class can do, instanceof won’t help so you’re left with either making extra functions like bool isAggregating(classname) or reflection.   Admittedly magic is nice, and even if a isAggregating() method were to solve all the problems, I would strongly advise to stay away from such temptations.

To quote Matt Zandstra,

Magic is nice but clarity is much nicer.

Downloads

aggregation.zip
Here’s a zip file containing all the classes mentioned above plus support for properties.   It’s perhaps worth mentioning that this really isn’t production-ready, should you ever really want to use such a thing it will need a few tweaks first – namely how to handle adding two classes which define methods of the same name – which one has priority?   No doubt you’ll also want some inspection methods such as isAggregating() as previously mentioned.

Posted in Object oriented programming, PHP | Tagged , , , , , , , | 5 Comments

PEAR Cache_Lite – efficient group cleaning

After using PEAR Cache_Lite for a while, we began to notice that as traffic increased the web servers spent more and more time thrashing their discs. On closer inspection we noticed that the servers were pretty much constantly parsing the entire cache directory structure.

Whenever you call Cache_Lite::clean() to remove a group of cached elements, it parses the entire cache directory structure looking for cache files which have the correct group hash in the filename. This was problematic for us because we stored a lot of data in groups, for example with messages – each page of messages for each user was stored in one group. Whenever someone sends a message, the system then deletes the cache group containing the recipient’s messages. As the cache directory structure increased in size, it took longer and longer to parse, and with increasing traffic the web servers were soon doing nothing but parsing the cache directory.

The solution I came up with was to prepend the name of each group with a number which was also cached. So when a request arrives for a cached item in group “messages”, the cache system looked for the cached group identifier number and prepended it to the group name, resulting in an internal group name like “1234_messages”.

The overhead is an extra cache “get”, but the advantage is that in order to expire a whole group you just have to increment the identifier number by one, (get, increment, save). So when the group is accessed again, the internal group name becomes “1235_messages”, which is not yet set, and so the application can regenerate the cache.

In my opinion this additional “get” is a price worth paying, especially as it’s a relatively very quick operation, and the time saved expiring a group is many times faster.

Finally

You might be thinking to yourself, “what about all those expired cache files just left on the disc?”. Well, we set a CRON job to run every day and delete all files older than 3 days. As none of the caches lasted longer than three days this was a safe duration.

In fact, we don’t use disc caching anywhere near as much as we did, now we use Memcached for most things, but for small and often used caches (such as IDs) we still use the disc cache as it’s by far the fastest.

Posted in PHP | Tagged , , , , , , , , , | Leave a comment

PEAR Cache_Lite – preventing stampeding

The PEAR Cache_Lite package is an excellent caching system; lightweight and fast, however when put into use on a high-traffic website a few issues came to light. The first problem we hit was stampeding.

What’s stampeding?

Stampeding is the situation when a request, let’s say from User1, arrives for a cached item that has expired. The cache system returns boolean false and the process of rebuilding that cached data begins, calling the database, formatting the data and so on.

If, during this process of rebuilding the cached data, another request arrives for the same cached item, let’s say from User2, another process of rebuilding the cached data begins. This is because the process started by User1 has not yet finished and so the cache system still returns boolean false when requested for the cached item.

So now we have two processes running, regenerating the same cache item. The situation can get out of hand if more and more requests for the same cache item arrive – causing the load on the web server or database to increase, and everything to potentially grind to a halt.

The solution

What’s required is for the cache system to know that a particular cache is being regenerated and therefore return the old cache until the new data has been regenerated. Thankfully this can be achieved very simply with the addition of just one extra line of code into the Cache_Lite class, Lite.php.

The trick is to touch() the cache file immediately after realising it has expired in the Cache_Lite::get() function. After touching the file, the get function will return false and the calling code will regenerate the cache data.

@touch($this->_file);

By touching the file, the modification time of the cache file is set to the current time and therefore all subsequent requests will think the cache is still valid and return the old data. Once the first process has regenerated the data, it saves it and the cache file once again contains up-to-date data.

Some finishing touches

By touching the cache, processes immediately following the one which is regenerating the fresh data will return out-of-date data, albeit by a matter of seconds – which in most cases really won’t matter nor be noticed.

If, however, something were to happen to the process regenerating the data, such as an uncaught exception, database timeout, etc that it would fail and not save the cache, then the old cache will be valid until it expires again – so it will have effectively been valid for twice its intended lifetime.

We can limit this by setting the modification time in the touch command to be the current time, minus the cache lifetime, plus 60s – which would mean that if the regenerating process were to fail, the cache would only be valid for another 60s.

@touch($this->_file, time() - abs($this->_lifeTime) + 60);

Posted in PHP | Tagged , , , , , , , | 3 Comments