The current system

I arrived at the ILC at the beginning of December. I started off just watching and admiring the work of other volunteers but eventually had to take a class on for myself. The then current system of record-keeping and proficiency-recording was paper-based. There were folders for each class and within each folder, booklets for each student.

Each booklet recorded a single student’s reading level, their strengths, their weaknesses and any concerns previous teachers had. These booklets would inevitably become full of volunteers’ notes. So many volunteers… so many notes.

Many of the scribblings were not dated and not particularly tidy. Lesson plans were recorded on paper too, but only as a single bullet point per lesson.

Don’t let me paint a bad picture though. The work being done here is and was always fantastic! The typical volunteer would plan a lesson the day before and record the plan on their own stationary, whether that be a notebook or a scrap of paper. Despite my reservations about it being paper-based the lesson’s still got planned and the students still picked up valuable language skills.

The problem, as I saw it, was merely one of efficiency. The case for a computer-based alternative was, I felt, rock-solid. I begun.

The new system

So, for the last few weeks I’ve been working on a new web application for the Isara Foundation which holds student records, lesson plans, teacher profiles and makes all this data accessible through an intuitive interface. Current volunteers can leave their lesson plans, student notes and class notes on a centralised system. New volunteers come along and don’t need to worry about digging through masses of paper to get the 101 on their assigned class. They can see all the data they could possibly want before even stepping into the classroom:

The new system, which is already being used (I guess it’s beta), has the following features:

• Stores basic data about students. Their names, ages, addresses, etc.
• Stores photos for all students. Great for learning your students’ names!
• Stores teacher profiles and allows teachers to be assigned to classes and vice-versa.
• Allows teachers to leave notes on student profiles (individual strengths/weaknesses/comments) and class profiles (comments about class in general).
• Allows teachers to enter their lesson plans, and look over previous lesson plans.
• All students’ skill levels recorded, which will eventually enable us to draw some interesting statistics about teaching techniques and skill acquisition.
• Makes it possible to receive notifications about student birthdays! (my favourite feature)

In essence, it is a primitive system. Nothing too fancy is going on, but I definitely enjoyed building it!

As some of you will know, I used CodeIgniter for this project, simply for its insistence on MVC and the fact that I was familiar with its API.

I don’t regret my decision to use CI. It has worked perfectly.

Implementation notes

The database schema seemed simple at first, but then came normalisation — and with that, apparent complexity. Here’s the schema! (not including audit tables)

As one is supposed to, I created separate models for every table, except the auditing ones. I created MY_Model which took care of the auditing and added a few useful methods. All of my actual models inherited from this.

The models were originally used for creating, reading, updating and deleting, but I eventually moved over to a structure which delegated to regular models for creating, updating and deleting, but utilised a specialised query model for the reading. I did this because I originally found that I was duplicating a lot of code within models, and decided to abstract the whole querying business to its own model. This has worked remarkably well and has facilitated a much simpler querying syntax and interface.

Within Query_model I can specify what needs to be selected and from where within an associative array:

// ...
'classes' => array(
    'select' => "
        classes.*,
        class_slots.name as slot_name,
        classes.name as class_name,
        class_slots.days as class_days,
        class_slots.time as class_time,
        rooms.id as room_id,
        rooms.name as room_name,
        rooms.photo as room_photo,
        COUNT(DISTINCT students.id) as student_count,
        GROUP_CONCAT(DISTINCT CONVERT(teacher_assignments.teacher_id, CHAR(8)) SEPARATOR ',') AS teacher_ids,
        GROUP_CONCAT(DISTINCT teachers.name SEPARATOR ',') AS teacher_names		
    ",
    'from' => "
        classes
            LEFT JOIN teacher_assignments
                ON classes.id = teacher_assignments.class_id
                AND teacher_assignments.active = TRUE
            LEFT JOIN class_slots
                ON class_slots.id = classes.slot_id
            LEFT JOIN teachers
                ON teachers.id = teacher_assignments.teacher_id
            LEFT JOIN student_assignments
                ON student_assignments.class_id = classes.id
            LEFT JOIN students
                ON students.id = student_assignments.student_id
            LEFT JOIN rooms
                ON classes.assigned_room_id = rooms.id
    ", 
    'where' => "",
    'groupBy' => 'classes.id'
),
// ...

And of course, I store the the fields that one would want to query by in a separate array:

// ...
'classes' => array(
    'teacher_id' => "teachers.id %c AND teacher_assignments.active = TRUE",
    'student_id' => "students.id %c AND student_assignments.active = TRUE",
    'id' => "classes.id %c",
    'room_id' => "rooms.id %c",
    'has_note_id' => "notes.id %c",
    'slot_id' => "class_slots.id %c"
),
// ...

I love this kind of low-level control!

The %c token you see above is replaced with an operator and the value to be searched. Operators are stored like so:

var $operators = array(
    'gt' => '>"%v"',
    'lt' => '<"%v"',
    'gte' => '>="%v"',
    'lte' => '<="%v"',
    'e' => '="%v"',
    'ne' => '!="%v"',
    'c' => 'LIKE "%%v%"',
    '>' => '>"%v"',
    '<' => '<"%v"',
    '>=' => '>="%v"',
    '<=' => '<="%v"',
    '=' => '="%v"',
    '!=' => '!="%v"',
    'contains' => 'LIKE "%%v%"',
);

The first bulk of operators you see there (gt, lte etc.) are utilised within query strings. E.g.

/query?ret=students&k=age&c=gte&v=16
(get all students with an age >= 16)

Don’t worry though! This seemingly low-level access isn’t actually that revealing. In order for a query to execute, the field that you want to query by must be specified, in the query-model, as must the operator. Values are escaped also.

I want to share more about the implementation but it turns out everything else is quite boring and basic. The query-model thing was really the only point worth nothing. That said, I am trying out an action-managing idea which allows delayed database actions such as deletion or flagging. Each action has a UUID and can be activated via a URL (this would also work well for email-activation links). Anyway, maybe next time…

It works!

We’ve been entering data and testing it for about two weeks and today saw a mass adoption by all volunteers at Isara. Hopefully this system will last, at least until a more efficient alternative comes along.

Anyway, as should be well known by anyone who frequents this blog, I am currently volunteering in Thailand. Specifically, I am working with Isara, an awesome non-profit foundation based in Nong Khai.

Let me try and shatter what you think you know about Thailand and put across what I know thus far in a paragraph. There are no blanket statements I can make such as “the food is delicious” or “the people are kind, generous and beautiful” because such things would only be as true as if said of any other country in the world.

Isaan is Thailand’s poorest province — this is where Nong Khai is situated. The average farmer makes around 140 Baht each day, which is about £3 ($5), and considering everything, people from this area seem contented with their lives. Farangs (foreigners, specifically white ones) are almost revered for just being white. The locals find novelty in our white skin, our coloured hair and eyes, our height and our language. Children here are demotivated by the government education system which seems to attract dispassionate teachers because of it being one of few jobs with social security benefits. The children, albeit demotivated, are enthusiastic when inspired and willing to learn when shown what they can become. There are struggles. Uneducated people pollute and abuse the environment and each other. Health, safety and true education come second to saving face and appearing calm and subdued. The people here cover every spec of the spectrum — there are all kinds. There are many farang here too… some older folk, some younger. Some looking for their Thai brides, some travelling, and some staying to help. Life here is cheap for a westerner. Life here is hard for a local. Learning English is a way out, an opportunity to become something more, an opportunity to take the world by the horns and do what you will!

I’m currently working on developing a computer-based system for Isara’s learning center, where local people are invited to learn English for free three times a week. Currently all student records are kept in a paper-based system — this includes details about their progress in class. Moving it to a computer-based system will waste less paper, provide a centralised store of student details and progress, enable querying of the data to determine which students are behind and what topics have been taught, and it will provide a wealth of query-able, filter-able, process-able data that could save many man-hours of paper-fiddling and printing!

It’s still a plan in progress, but the DB schema looks like this thus far (all ‘_log’ tables are for auditing). I am going to be using CodeIgniter, just because I remember how ridiculously simple it made a previous project. If you haven’t, you should try it out!

It insists upon separating your concerns into the traditional M-V-C pattern (Model, View, Controller) and has a bunch of auto-load-able helper classes and libraries. It’s very widely used, so finding help online is rarely a problem (Google is your friend).

Got any tips or stories to share about CI? Please share.

• The string to be wrapped.
• The column width (a number, default: 75)
• The character(s) to be inserted at every break. (default: ‘\n’)
• The cut: a Boolean value (false by default). See PHP docs for more info.

The code

function wordwrap( str, width, brk, cut ) {
 
    brk = brk || '\n';
    width = width || 75;
    cut = cut || false;
 
    if (!str) { return str; }
 
    var regex = '.{1,' +width+ '}(\\s|$)' + (cut ? '|.{' +width+ '}|.+$' : '|\\S+?(\\s|$)');
 
    return str.match( RegExp(regex, 'g') ).join( brk );
 
}

Usage

wordwrap('The quick brown fox jumped over the lazy dog.', 20, '<br/>\n');

The quick brown fox <br/>
jumped over the lazy <br/>
dog.

I know there are other solutions out there; the ones I saw seemed a bit slow though, not to mention unnecessarily complicated/bloated…

// (C) Copyright JAMES PADOLSEY
class tweetFromFile {
 
    private $curlHandle;
    private $updateFile;
    private $archiveFile;
 
    private function getNewStatus() {
 
        $upcomingTweetsFile = $this->files['upcoming'];
        $archivedTweetsFile = $this->files['archive'];
 
        $upcomingTweets_R = fopen($upcomingTweetsFile, "r");
 
        // Get upcoming Tweets:
        $contents = fread($upcomingTweets_R, filesize($upcomingTweetsFile));
        $splitContents = preg_split('/\n/', $contents, 2);
 
        // ARCHIVE OLD POSTS:
        $archive = fopen($archivedTweetsFile, "a");
        fwrite($archive, $splitContents[0]."\n");
 
        // Remove top line from upcoming:
        $upcomingTweets_W = fopen($upcomingTweetsFile, "w");
        fwrite($upcomingTweets_W, $splitContents[1]);
 
        // Clean up
        fclose($upcomingTweets_W);
        fclose($upcomingTweets_R);
        fclose($archive);
 
        return $splitContents[0];
    }
 
    public $files = array('upcoming' => '', 'archive' => '');
 
    public function __construct($username, $password, $filename) {
 
        $this->curlHandle = curl_init();
        $this->files['upcoming'] = $filename;
        $this->files['archive'] = 'ARCHIVE_' . $filename;
 
        // Shortcut:
        $ch = $this->curlHandle;
 
        curl_setopt($ch, CURLOPT_URL, "http://twitter.com/statuses/update.xml");
        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_POST, 1);
        curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
 
    }
 
    public function __destruct() {
        curl_close($this->curlHandle);
    }
 
    public function updateStatus() {
 
        $status = $this->getNewStatus();
 
        curl_setopt($this->curlHandle, CURLOPT_POSTFIELDS, "status=$status");
 
        $result = curl_exec($this->curlHandle);	
	$resultArray = curl_getinfo($this->curlHandle);
 
        if ($resultArray['http_code'] == 200) return true;
 
        return false;
 
    }
 
}
 
// ==================
// ===== USAGE ======
// ==================
 
$tweet = new tweetFromFile('twitterUsername', 'password', 'textfile.txt');
$success = $tweet->updateStatus();
if ($success) {
    echo 'Twitter updated!';
} else {
    echo 'Hmm, an error...';
}

The class will retrieve each new update from the specified text file. Each update needs to be on a new line; an example text file:

Blah blah blah, this is the first update...
And another update!
Third and last update!!! ARGH!

The PHP class will remove each update from this file when it’s added to Twitter so all you need to do is keep adding to the bottom of the file…

This won’t do the dripping automatically. You’ll need to setup a ‘cron job’ to request the script at selected intervals (e.g. every two hours). I’m rubbish at Apache and anything to do with the command line so I haven’t quite accomplished this part yet… maybe someone else wants to give it a go?

Even with getTimezoneOffset() you’re still relying on the client!

It turns out that the only way of getting a complete and accurate time from any timezone requires a little bit of server interaction. Luckily there are a few APIs out there that offer this service (actually, there’s only one; well I couldn’t find any others)!

Here we’re using the ‘json-time’ API developed by Simon Willson:

JSON-time

function getTime(zone, success) {
    var url = 'http://json-time.appspot.com/time.json?tz=' + zone,
        ud = 'json' + (+new Date());
    window[ud]= function(o){
        success && success(new Date(o.datetime), o);
    };
    document.getElementsByTagName('head')[0].appendChild((function(){
        var s = document.createElement('script');
        s.type = 'text/javascript';
        s.src = url + '&callback=' + ud;
        return s;
    })());
}

Usage

The first parameter of the callback function is the time (having been passed through the Date constructor), so you can use Date methods such as getSeconds() etc. The second parameter is the entire object returned from the JSON request.

// Alert GMT:
getTime('GMT', function(time){
    alert(time);
});
 
// Get London time, and format it:
getTime('Europe/London', function(time){
    var formatted = time.getHours() + ':' 
                  + time.getMinutes() + ':'
                  + time.getSeconds();
    alert( 'The time in London is ' + formatted );
});

For reference, here’s a list of all the available timezones (to use one, remove the backslash).

jQuery version

$.getTime = function(zone, success) {
    var url = 'http://json-time.appspot.com/time.json?tz='
            + zone + '&callback=?';
    $.getJSON(url, function(o){
        success && success(new Date(o.datetime), o);
    });
};
 
// Usage:
$.getTime('GMT', function(time){
    alert(time);
});

MooTools version

You’ll need to download the JsonP class.

function getTime(zone, success) {
    var url = 'http://json-time.appspot.com/time.json';
    new JsonP(url, {
        data: { tz: zone },
        onComplete: function(o){
            success && success(new Date(o.datetime), o);
        }
    }).request();
}
 
// Usage:
getTime('GMT', function(time){
    alert(time);
});

Thank you Simon Willison for making this API; seriously, I searched around the entire internets just for one damn API and I almost lost hope!

There are two components to this: the PHP script and some simple JavaScript:

The Code (PHP):

/* PHP file, e.g. scanImageDirectory.json.php */
 
// Check that a callback function has been specified:
if (!isset($_GET['callback']) || !isset($_GET['directory'])) exit;
 
// Use PHP5's scandir function to scan all
// of images directory:
$dirContents = scandir($_GET['directory']);
 
// Define function to confirm each
// filename is a valid image name/extension:
function isImageFile($src) {
    return preg_match('/^.+\.(gif|png|jpe?g|bmp|tif)$/i', $src);
}
 
// Loop through directory files and add to
// $arrayContents on each iteration:
$arrayContents = '';
foreach($dirContents as $image) {
    if (isImageFile($image)) {
        $arrayContents .= !empty($arrayContents) ? ',' : '';
        $arrayContents .= '"' . 'images/' . $image . '"';
    }
}
 
// Prepate JSON(P) output
$output = $_GET['callback'] . '({\'images\':[' . $arrayContents . ']});';
 
// Output the output:
echo $output;

The above PHP script will use PHP5′s scandir to scan the ‘images’ directory and it outputs a list of all images in JSON format. (so it can be used by our JavaScript). The JavaScript is below:

The Code (JavaScript):

/* JavaScript Component */
 
function preloadImagesFromDirectory(dir) {
 
    if(!dir) return;
 
    function getJSON(URL,success){
 
        // Create new function (within global namespace)
        // (With unique name):
        var uniqueID = 'json'+(+(new Date()));
            window[uniqueID] = function(data){
                success && success(data);
            };
 
        // Append new SCRIPT element to DOM:
        document.getElementsByTagName('body')[0].appendChild((function(){
            var script = document.createElement('script');
            script.type = 'text/javascript';
            script.src = URL.replace('callback=?','callback=' + uniqueID);
            return script;
        })());
 
    }
 
    function preload(srcArray) {
        for(var i = 0; i < srcArray.length; i++) {
            (new Image()).src = srcArray[i];
        }
    }
 
    // Get that JSON data:
    getJSON('scanImageDirectory.php?directory=' + encodeURIComponent(dir) + '&callback=?', function(data){
        return data.images ? preload( data.images ) : false;
    });
 
}

The above JavaScript uses JSONP to load the PHP file into a SCRIPT element as if it were a JavaScript file itself, then the automatically generated function is fired which then executes the specified callback (‘success’) function which is specified when calling getJSON. The callback then preloads the passed data (i.e. the array of images).

Usage:

// We don't want to disturb anything so we'll wait
// until everything's done loading before preloading:
window.onload = function(){
    preloadImagesFromDirectory('images/');
}

Security concerns:

Since the directory to be scanned is specified in a query string it would be VERY easy for ANYONE to gain information on your file structure/nomenclature. The regular expression used to validate the image names on the server-side (the PHP script) will stop any files from being listed that do not end in gif, png, jpg, jpeg, bmp or tif. If you’re still concerned then simply specify the directory to be scanned from within the PHP script instead of having it passed through the query string.