What I'm going to do with my Raspberry Pi

    That magic email from Farnell came on Saturday, my Raspberry Pi is in the post!
    So, what am I going to do with it?
    Like loads of other geeks I expect I'll plug it into my telly, connect it to my router and use it as a web terminal and media centre with geek bragging rights. There it'll sit for however many years it takes until I get a new telly or a Raspberry Pi 2, unseen and uncomplaining. But rather wasted, don't you think.

    Somewhere, the Flying Spaghetti Monster has just killed a kitten.

    So what do I really want it to do? I will have a very small and moderately powerful computer - insanely powerful by the standards of a few years ago -  that uses negligible electrical power and can be left on all the time. I'm still going to plug it into my router and telly, but to make it earn its keep I'm going to have it run my keyword analysis tool.
    Events have moved on a little since my blog post describing the tool, but the principle is still the same. I take new posts every day from a big list of RSS feeds and process them for keyword phrases which I store in a database. I can then extract frequencies and collocates over time, which gives me a picture of the interrelationship between the language and terms in the news over any given period. It not only fulfils my original aim of having a tool that would generate keywords and phrases for previously unseen search terms, but also allows any newsworthy subject to be examined in a way that is not possible by any other means.
    The original tool runs in PHP on my Windows laptop. Its MySQL database is pushed well beyond its limit, and I have been working on a version that uses a large directory tree of precomputed JSON files instead. It's an approach I've since also used in my work, relying on the principle that disk space is cheap and quick while complex joins on monster MySQL databases are expensive and slow.
    I could of course compile PHP for my Pi. It's probably already available precompiled anyway. But the Pi's a Python platform (Try saying that after five pints of real cider!) and that offers me a unique opportunity. My PHP code does the job, but it relies on my own language processing libraries which I built myself as a search engine specialist. I'm not a computational linguist so I'd be the first to say that they aren't as good as they could be.
    Python has the incredibly useful Natural Language Toolkit libraries which allow me to do so much more with my source texts, and so much more quickly than my PHP code. So my first effort with my Pi will be to port my keyword tool to Python, using the NLTK instead of my own library. The Pi will still sit behind my telly and be used for the occasional bit of web surfing, but for the rest of the time it'll be crunching keywords and giving me lots of lovely language data that I can work with in real time rather than with enough time to make a cup of tea every time I make a MySQL query.
    In a way I'm not taking advantage of everything the Pi can do. Almost any internet connected computer could do this job, I'm only using a Pi because it's cheap and low power, and I've lusted for one ever since I read their early press releases. Other people will use the Pi's hardware capabilities to do much more eye-catching things. But my Pi, quietly crunching words all day and night behind my telly, will still be earning its keep. It will allow me to learn new things and since its data is likely to end up in some of my work stuff it may even in its own small way make a contribution to the wider understanding of language.
    So that's what I'll be doing with my Pi, what'll you be doing with yours?

FAILBlog #fail

    My life is complete. A piece of software I wrote for the Oxford Words Blog, an online Shakespearean vocabulary checker, has been featured on FAILBlog.
    Pretty funny, the FAILBlog submitter has picked up on the fact that if you put the word 'balls' into the form repeated more than 20 times it tells you your English is 100% Shakespearean. Which though undeniably entertaining is entirely true, since 'balls' was a word used by the Bard.
    Unfortunately though it is not a fail, unless you didn't read the page and have misunderstood the purpose of the script. It is not there to deliver a verdict on the quality of writing, merely to correlate the vocabulary with that of the Bard. Which it has done, exactly as it should.
    I can't help feeling FAILBlog have failed though so, I posted the following comment:

First of all, thank you FAILBlog for featuring this. It has always been an ambition of mine to have a piece of my software featured on your site, and now I have achieved that goal.
I wrote the script featured in the screenshot above, you can find it at http://blog.oxforddictionaries.com/2011/08/how-shakespearean-are-you/.
At first sight it does look like a fail, but as it says on the page this is not a script for judging the quality of writing, it is simply a vocabulary correlation checker that compares the vocabulary entered with that of the Bard and delivers a percentage result. So since the Bard did use ‘balls’, it is exactly right in saying the text is 100% Shakespearean in vocabulary, because it is.
But that does not make the odd-looking result any less entertaining, so please feel free to have a laugh at our expense.

     I guess the guys at FAILBlog don't appreciate it when they get something wrong and the originator makes a comment about it, because their moderators have chosen not to publish my comment.
   
Edit - several hours after I posted this piece and hours after publishing a load of other comments the FAILBlog people published my comment. Well done! Did I stumble on an editorial approval policy?:)

A dictionary search box for your website or blog

The Oxford Dictionaries Online search box widget I mentioned in a previous post has now gone live and is available for you to put on your own blogs or web sites. It is available in two versions, one for each of world and US English, and an instance of the final result can be seen at the top of the right hand column of this blog.

Get it here: http://oxforddictionaries.com/page/searchbox

2010 as viewed through Daily Express headlines

    A couple of years ago I was working in a small business centre in Oxfordshire, a building that was home to several small tech companies. We had a common room with a kitchen and dining area, and the building management provided us with a selection of daily newspapers.
    Among those papers were a high-end broadsheet, a red-top tabloid and the Daily Express. I became fascinated by the Express headlines over the year, and resolved to collect them, initially just for the ones barmy enough to make me laugh but later all of them for a collective analysis. This post is the result, an infographic that tries to capture the essence of a year as seen on the front pages of the Express.
    So why did the Express fascinate me? Here is a gloriously barmy  newspaper obsessed with cancer stories, lurid tales of illegal immigrants, and the UK housing market, using a colourful vocabulary all of its own to stir its readers into righteous anger over stories that very often have little relationship to the main news stories of the day. Everyone is furious or outraged in Express-land, we're under siege from miscellaneous foreigners and there is no Government decision or economic movement that is not designed to let down the Express reader. This continual negativity is offset with ludicrous non-stories proclaiming miraculous rises in the housing market or turning tentative conclusions in scientific papers into medical breakthroughs in the fight against cancer. Seeing what they had on the front page became part of the entertainment of my morning routine, there is nothing like a ludicrous headline over your morning coffee for starting the day with a laugh.
    The infographic shows some of the pretty pictures that can be gleaned from a year's Express headlines. A PNG version can be downloaded by clicking on the image or you can download it as a PDF from this Google Docs link. I hope you enjoy reading it as much as I enjoyed making it.

A company: "Them", or "It"?

Consider the following phrase: 

"Since it is the market leader it can be assumed that what works in relation to Google will also work for its competitors" 

     Google, like any other company, is an entity. "It". But Google is also a collection of offices full of people. "Them". 
     "Google are writing software that..." or "Google is writing software that..."? My inbuilt English language parser tells me to use the former. Is it leading me astray?

My compliments to the cook: SEO vs. SEM

    When I was a small child I attended a primary school in an English village. Summers were long and hot, there were jumpers for goalposts and our school meals were awful. They were the creations of the school cook, a rather nice lady whose culinary output was probably stunted by a poor budget and the dead hand of Ministry of Education dieticians. It was with great surprise then when I moved to secondary school that I found the meals were rather good, worth looking forward to in fact, for they were assembled not by a cook, but a chef. With a white hat and all, very impressive.
    My profession is usually referred to as search engine optimisation, often represented by the initialism SEO. You will rarely see either in my personal lexicon, instead I prefer search engine marketing.
    My reasons for this are twofold: to give a sense of the wider task involved in helping a web site to increase its visibility in the search engines through legitimate means and to differentiate myself from the work of the blackhats in the gutter of my industry. A few years ago while contracting as a quality rater for the large search engine you probably use daily I spent a lot of time following up keyword stuffed link farms, valueless spam blogs and hidden or misleading rubbish from people who definitely refer to themselves as being in the SEO business, so for me the distinction is an important one. I'm lucky enough now to work in-house at a large publishing business and need never ply my trade further afield, so I see no reason to associate myself with the term SEO.
    Looking at a Google Insights search comparing the two terms I find I'm at least not entirely alone. Search engine marketing is used about half as much as search engine optimisation(or optimization for a US search) but it's still a significant enough term for me to be able to describe myself thus without blank looks. Because as with the school catering staff of 1980s Oxfordshire, I'd rather be a chef than a cook.

All blogs have to start somewhere

Keyword

1. a word which acts as the key to a cipher or code
2. a word or concept of great significance

Geek

1. an unfashionable or socially inept person
2. [usually with modifier] a knowledgeable and obsessive enthusiast

    This is obviously one of those cases when you wish you hadn't looked a word up in the dictionary. I'm a search engine specialist by trade, so "A knowledgable and obsessive enthusiast for words of great significance" doesn't sound too bad. I'm not so sure about "An unfashionable or socially inept person" though.

    I enjoy my job, and in doing it over the years I have frequently encountered words, phrases, techniques and bits of code that have made me think at a tangent to what I am being paid to do. These tangents sometimes stick around in my head for a while, and this blog represents a long-overdue outlet for them.