Making sure my data is readable in a hundred years

Having spent some twenty years researching my family history, I obviously want to make sure that the fruits of my work are accessible to the generations that follow, so how do I ensure that it is all readable in a hundred years?
When I started my research, in the days before PCs, Macs etc, a colleague invested in a Philips Videowriter – basically a huge CRT-based box with built-in thermal printer. It could perform just one task – word processing – using a proprietary format and a 3.25″ (yes, 3.25, not 3.5″) drive. Within a couple of years, it died and could not be repaired. The disks were unreadable and, worst of all, all the hard copies had faded as thermal prints are wont to do. All the data – years of research – was lost. I am determined that this won’t happen to me.
The first thing to do is define what sort of data I am talking about. I think it can be divided into two categories:
Hard copy media – printed copies of research, certificates, old photos, etc
Electronic media – scans and source files, photos, databases, research notes etc
Hard Copy Media
Preserving old documents is a science in itself, so apart from scanning, covered below under electronic media, I won’t attempt to discuss that here.
However, I have produced a book containing biographies, research notes, images, photos and family trees – can I assume that it will survive? The problem is, modern toner-based laser prints on re-cycled, generic photo-copy paper are intended to be quick and cheap, not durable. No one really knows how long the toner will remain stable. Ten to twenty years shouldn’t be a problem, but beyond that?
Electronic Media
This subject has two specific aspects: format and storage.
Format
Twenty years ago, the standard word processor was Word Perfect and images were stored as 8-bit GIFs. Now, it’s more likely to be Word and jpegs and, along with Adobe’s PDF format, it is probably no exaggeration to say that there are literally billions of jpegs, pdfs and docs in existence, so even when the standards are superseded, it is likely that those files will be readable, even if only by libraries or specialists. Similarly, the family tree databases are also likely to be readable. Although each family tree program uses a different format and structure which can change from version to version, there is an industry standard specifically for family tree databases managed by the LDS(1). The format, GEDCOM, is ASCII-readable yet maintains names, facts and relationships.
The media on which it is stored, however, is a different matter.
Media
A few years ago, 3.5″ and 5.25″ discs would have been the norm, but now few people could read either. Since there are probably billions of CDs and DVDs in existence, it is likely that readers for them will exist in the future (even if only in libraries etc), but dye-based CD and DVD ROMs were never intended to last beyond ten years and it is thus unlikely that they will readable in a hundred years – I have already had some fail after 12 years.
PATA and SATA hard drives are already being replaced and USB2-based drives will die off for USB3 which will, in turn, go the way of SCSI, PCMCIA and Firewire. I also doubt that flash memory such as Compact Flash, SD etc will survive as a mainstream format for more than 20 years. Online storage, either using cloud-based virtual drives or hosting research on resources such as Ancestry are great…for as long as you pay the subscription or as long as the hosting company exists. Even if Ancestry survives a hundred years or, more likely, some other online repository is created in its place, how will anyone know our data is there?
Conclusion
It is clear that there is no perfect solution. For data formats, sticking with widely used standards makes sense, and I would encourage genealogists to regularly back up their databases in the Gedcom format. However, the only solution that is truly future-proof is to continually port the data into the new formats and media as they emerge.
(1) LDS – (The Church of JC and the) Latter Day Saints – vast resources employed in genealogy making them a key mover in genealogy technology – perhaps less so since the introduction of paid-for services such as Ancestry etc.
Posted in family history, genealogy, Technology | Tagged , , | Comments Off on Making sure my data is readable in a hundred years

No more Censuses… Censi?

There was an interesting article in some of yesterday’s papers which suggested that next year’s national census will be the last. There seemed to be three reasons given:

Firstly, Britain’s population is now highly dynamic, with economic migration perhaps likely to become more prevalent. Secondly, there seemed to be concern that, whether due to paranoia, deception or the great British sense of humour, there were too many false or joke responses in recent censuses. Remember back to the 1991 census at the height of the Poll Tax fiasco when we feared our census returns would be cross-checked against our tax declarations? Then, in 2001, we were asked for the first time to declare our religious beliefs – 390,000 of us (yep, I do mean “us”) declared ourselves as “Jedi”.

The third reason given was that there are now many better ways of judging the size and make up of the population. Simply asking Tesco’s for their Club Card data would be start (although I doubt the State could afford to buy it from them), while a bit of data mining on Facebook or the viewing figures for X Factor would help.

So, all this got me thinking about the implications of discontinuing the censuses on future genealogists. Censuses have been taken in the UK since 1811 but only those from 1841 were kept. 1941 didn’t happen because of the war and 1911 is the latest one to be released for public research. Online resources such as Ancestry and Find My Past have fully indexed searchable database of all the accessible censuses and they are the most fabulous window into our ancestors’ lives – telling us where they lived, their family structures as children were born, grew up and left home, their occupations and where they were born.

In a hundred years then, what will our descendents be able to find out about us after the 2011 census? Think about it, our medical records will soon be available on the NHS network, and will no doubt be published in a hundred years, as will criminal records and pretty much all public records on which we appear.

What about our online personas? Facebook already retains profiles for people who have died. While I doubt Facebook will be around in 20, much less 100 years, what will happen to the data? Even if it was taken offline, the data won’t (probably can’t) be destroyed. Will genealogists force its custodians to release the data under Freedom of Information? What about the other companies or agencies that hold databases on us – Tescos, Google, O2 et al, MI5? How about our blogs?

So, yeah it looks like future genealogists will have plenty of information on us with or without the national census. Scary, isn’t it?

Posted in family history, genealogy | Tagged | Comments Off on No more Censuses… Censi?

What have Toyota done to upset the BBC?

So what exactly have Toyota done to upset the mighty BBC? I can’t help wondering whether one of the Trustees or Governors has had a falling out with their local service centre, because the Beeb are pulling no punches, are they?

Ok, Ok, there have been some issues with the sticking accelerator pedals which has caused “around 20” incidents in the US over the last few years. Alright, Toyota could have reacted sooner to invoke a simple re-call, but car manufacturers issue recalls all the time, often for more serious defects. So why have the BBC led on the story for at least three days, using words like “crisis” and even (I still cannot believe this) sending an outside broadcast crew this morning to a Toyota dealership to film – live!!!- one of the fixes being implemented.

And now, of course, we have the smug sniggering about the Prius, the car that the BBC, in the shape of Top Gear, put down at every opportunity. No, I don’t have one, but I can appreciate the technology behind it and the fact that, like it or not, if we want to carry on driving individual cars, then hybrids and electric cars are the future.

I await, with bated breath, what new angle the BBC will find for tommorow’s lead. A expense scandal by a Toyota exec?

Posted in Rants & Comments | Comments Off on What have Toyota done to upset the BBC?

On the Ipad

Now I don’t like to be part of a crowd. I don’t follow any trends, run with any packs or, I’d like to think, don’t fall for the hype.

And yet, I couldn’t ignore the Ipad completely, could I?

So, the first surprise is that it is a super-sized Ipod Touch or Iphone, depending on which you choose.

That’s it really.

It runs the same apps, it looks the same but bigger, and it runs the same operating thingy.

And, you know what? That’s why it will be a staggering success. Because 70 million (70 million!) of us already have the baby version, love it, know how to use it, and have wondered why our “proper” computers couldn’t be as good.

Think about it: web browsing, video watching, music, games – yep, that’s about 95% of the use my laptop gets, tethered to the charger. And then there’s the document reader. Oh, how wonderful to have a screen shaped like a sheet of paper on which I can read books, active news sheets, reports and hold it like I would a book!

Ok, like many people, I joined in the live event and was a little disappointed… until I thought about it and realised that Apple have got it just right.

Do I want one? Yes, oh yes.

Posted in Technology | Tagged | Comments Off on On the Ipad

On “Postage & Packing”

I’ve been reading a lot of books, papers and blogs recently about online marketing. It’s probably true that many sites have really nailed the whole user experience thing. Sites such as Amazon are great at getting us to them – their reputation, our loyalty, high search rankings. And when we are on-site and we find what we want, there is plenty of reassurance that it is the right product – photos, descriptions and, especially, user reviews. We trust these sites. We trust them not to sell us fakes, or skim our credit card or sell our data to fraudsters.

But they are all rubbish when it comes to delivery.

I guess we all realise that “postage & packaging” is really a stealth mark-up. Somehow, we convince ourselves that if it isn’t included in the advertised price then we don’t need to include it in the cost of purchase. So, we get charged an extortionate price for them to put the goods in a cheap cardboard wrap and send it in the post. You want it in three days rather than five? Sure, that’s double. Overnight? Yippee, more profit.

And why can’t they be upfront about the dreaded p&p? Why wait until the very end of the buying process to add the delivery tax? Amazon have a particularly good scam: buy three things in the same basket, unwittingly from three different suppliers (when you thought you were buying from Amazon!) and pay three delivery taxes. Oh joy unbounded!!

And then comes the ritual of doorstep delivery. Despite the fact that they remain stuck in 1960s industrial relations, hell-bent on self destruction, the favourite deliverer remains the Royal Mail who now deliver once per day at noon.

When I’m not in.

So, armed with my “sorry I missed you” card, I have to wait until Saturday morning, drive to the central Post Office (ten miles round trip, say two quid in diesel), park (add another two quid for minimum of an hour) wait in line, show an ID and collect my goods.

What!!! I pay through the nose for postage and packing, then I have to pay more to go and collect it?

Look, if you’re reading this and you work for an online retailer:

– Show me the price inclusive of reasonable delivery – next-day great, day after ok, anything longer forget it.

– Delivery is not the same as dispatch. I don’t care when you dispatch it, I’m only interested in receiving it.

Posted in Rants & Comments | Comments Off on On “Postage & Packing”

Dear Steve Jobs

Dear Steve Jobs

I’d like to say thank-you for my Iphone. No, it wasn’t a Christmas present, I’ve had it since January 2009 and I LOVE IT. I just haven’t gotten around to thanking you yet.

However, the thought occurs that my contract with O2 expires in July, so that would be the perfect time for you to bring out a new model, wouldn’t it?

I was wondering though, if you could see your way clear to adding a couple of things.

For instance:

– Customised e-mail and text alerts because, frankly, the current ones are pants

– Synching over wi-fi; my computer connects wirelessly to the same network as my Iphone, so why do I need a piece of wire to get them to talk to each other?

– A better camera because, frankly, the existing one just isn’t anything. A better lens, zoom and a bit more resolution would be a nice.

– Better battery life. Yes, I know that the 3GS is better but still not great.

– An SMS outbox when for when there is no reception and automatic re-send it when there is a good signal.

Yep, that just about covers it.

Cheers

Pete

Posted in Technology | Tagged | Comments Off on Dear Steve Jobs

How to be a genealogist

Don’t get me wrong, I love “Who Do You Think You Are?”. It has done a huge amount to promote genealogy as a hobby which, in turn, has generated the market for sites such as Ancestry and FindMyPast to make a wealth of historical records online. It has shown millions that genealogy doesn’t have to be about lines of lineage stretching back hundreds of years but can be about what our fathers or grandfathers did in the wars – where and how they lived and what made them who they were.

I just want to check, though… hands up who thinks it’s as easy as it is on WDYTYA?

Anyone? Good.

I’m sure that, if you had the BBC’s resources and budget, you too could arrange for the curators of guilds and museums to greet you with open arms and to just happen to have a stack of records just waiting for you to “discover” them.

No, the reality is much harder, but getting easier all the time.

Genealogy is now one of the world’s most popular, and fastest growing past-times. Consequently, things are changing very fast with more and more books, magazines, TV programmes, websites etc dedicated to the subject, so please bear in mind that what follows may get out of date very quickly.

In the last ten years, the Internet, much as in everyday life, has become the primary tool and source of information for the genealogist… or should that be family historian? Is there a difference? If there is, I’d suggest that a genealogist is concerned with finding names, dates and places going back as far as possible – essentially, generating a pedigree. Meanwhile, the family historian is less concerned with going back a long way than they are concerned with finding out about their ancestors’ lives – where they lived and worked, war records, school days etc. Ultimately though, there is no difference. Most researchers start looking for names and dates and only begin to “flesh out” their stories as it gets harder to find anyone new.

There are three phases in the family history process, and I will describe each in turn. But first, a handy hint: record every search, positive or negative and document every scrap of information and every source. One thing about this hobby is that nothing happens in a hurry. If an ancestor is dead, he will still be dead in six months and scraps of information can take years to grow into a coherent story.

Phase 1 – Memories and shoe-boxes
The single most valuable sources of information are the memories of relatives and they should be exploited for all they are worth, but keep in mind the following:
• Do it now, before the memories are lost forever.
• Remember that names and relationships may be recalled accurately, but dates must be treated with caution
• Don’t expect Aunt Hilda to just tell you all she knows. Write out a series of questions beforehand, preferably ones which lead up to important details. Remember that some of this may be painful to her.
• Names and dates are all very well, but what you really need are anecdotes and photos. Remember to annotate photos straight away.
• Old family books, particularly bibles, can often be a good source of data as can wills, service records, passports and that old shoe box stuffed full of letters.

Phase 2 – Certificates and Censuses
It is important to have documentary proof before you can be certain about a relationship. The best of these are undoubtedly the General Records Office (GRO) certificates:

Birth certificates give name, birth-date, place of birth, parent’s names (including the mother’s maiden name) the father’s occupation and the parent’s address.
Marriage certificates reveal the names, father’s name and occupation for both parties. Also, an address at the time of the wedding.
Death certificates give the name and date plus the age of death and the name of the spouse. These rely on the testimony of whoever registers the death so must be treated with caution.

These certificates do not exist before 1837 when Lord Melbourne’s government introduced legislation under which all births, marriages and deaths had to be recorded and appropriate certificates issued. Copies of the records were kept locally at parish register offices and centrally at the GRO. Most, but not all, births, marriages and deaths were recorded until 1875 when it became an offence not to register an event.

FreeBMD is an ongoing project to index all the GRO registers. This makes it very easy to search for a year, quarter, volume and page reference for the event which can be used to order a copy of the certificate from the PRO online or through Ancestry.

Ten yearly censuses have been taken in the UK from 1801, but the 1841 census was the first to be kept for posterity. It is, however, incomplete and does not always record much more than the name of the head of the household and the number of people living there. Ages are usually rounded up to the nearest five. From 1851 onwards, however, the information is far more complete, giving the names, ages, occupation and place of birth for all people present at an address on the night of the census (usually during the first week in April). Households were visited by census enumerators to complete the census forms on the families’ behalf. This means, however, that we are at the mercy of the enumerators’ handwriting and spelling. The most recent census in the public domain is 1911.

Ancestry and FindMyPast both offer online, searchable indexes to all censuses up to 1901 (only FindMyPast holds the 1911 index) and it is very easy to find ancestors and download an image of the enumerator’s forms. In this respect, the 1911 census is different as it is the actual form, completed in the householder’s hand, which is available. Bear in mind, however, that to add to enumeration errors, there are a considerable number of transcription errors in the indexes.

Phase 3 – Everything Else
So, what happens when you have found all the ancestors born, married or who died after 1837? No more birth certificates, no more census returns and definitely no more personal recollections.

There are many possible sources of information, ranging from prison records, records of those transported to the colonies, lists of Catholics and Huguenots etc. The main sources are likely to be:

International Genealogical Index (IGI)
The Church of Jesus Christ of Latter Day Saints (LDS, aka the Mormons) believe that everyone who ever lived should be baptised as a Mormon, although to avoid controversy they keep to those already dead and not in a position to object. It follows that they spend a lot of time searching for individuals to baptise. They do this by transcribing local parish records into a massive database called the IGI which contains details of about 39 million individuals from England alone. As you might expect, some of the more conservative parishes have refused access to their records and the database is thus incomplete. For example, Essex is believed to be about 33% complete whilst Sussex is nearing 100%. The database can be searched on the Internet at www.familysearch.org. For a baptism or christening, the IGI gives an individual’s name, the date, the parent’s names and the church where the event took place. In the case of a wedding, the parent’s names are replaced by the Spouse’s name. From personal experience, it appears that entries can be found from about 1500 to around 1860. This is a very valuable source and I have found many ancestors from the IGI but it is definitely worth confirming anything with the original parish records.

Parish Records
Parishes were typically villages with a church and larger towns and cities would contain several parishes. Records of British baptisms, m
arriages and burials have been maintained by law since 1538 but not all clergymen kept proper records in the early years. The early baptism, marriage and burial records were usually jumbled together and some of them were written in Latin but by 1732 all registers were required to be written in English. For the first 200 years it was normal to record only the father’s full name and that of his child in baptismal. Most original parish registers are now available to view in County archives and have been microfiched. Some, but certainly not all, parishes are available online and some are available to buy on CD Rom. From 1598 the clergy had to send a copy of their entire year’s parish register to the local bishop. These copies, known as Bishop’s Transcripts are also available in County archives.

A note of caution is required here. Parish records, and consequently the IGI, do not record births – they record baptisms. Usually, children were baptised within a month of their birth but not always. It was also common to have a communal baptism. As an example, my second great-grandfather was baptised with four of his siblings in 1841, five years after he was born. Remember also that the most parishes were Anglican. Catholic, Jewish and later non-conformists churches collected their own records.

Commercial sites such as Ancestry are competing with each other to put more and more records online, such as military rolls, workhouse records, parish records from the London Metropolitan Archives, phonebooks and passenger records. These are incredibly valuable for fleshing out our ancestors’ lives. They also encourage members to upload their own trees but be aware that some genealogists may not be methodical as they should be, so don’t rely on other people’s research – check it yourself.

Posted in family history, genealogy | Comments Off on How to be a genealogist

On the Future of User Interfaces

What was the name of that movie? You know, the one with Tom Cruise?

“Minority Something”. Report, yes – “Minority Report”. He stood in front of this huge transparent screen and sort of dragged photos and video around like this (waves hand) and then he pulled the corners to make them bigger and then he…..
Spielberg apparently based that iconic sequence on conversations with Microsoft, so perhaps it is no surprise that Hollywood’s one-time virtual reality is now nearly real.

Microsoft recently created “Surface” a table-based computer with a horizontal screen that combines multiple, simultaneous touch inputs, gesture recognition, object and tag recognition and advanced graphics – and, yes, you can drag and resize objects like in the film. Surface is clearly targeted at multi-user interactivity: bars and restaurants for interactive ordering and playing; retail outlets for interactive catalogues and the corporate world for presentations and briefings. Powerpoint presentations will never be the same again.

These applications of Surface are rich in “ooh, that’s clever” moments, impressing with design and the user experience.

Apple’s Iphone brought gesture recognition to the consumer’s pocket and, almost overnight, the ubiquitous desktop and laptop looked slightly old-fashioned. There are, however, millions of personal computer users in the world and almost every one of them uses a keyboard and mouse.

Enter Windows 7 and Apple’s Snow Leopard, which support the growing number of multi-touch devices on the market. Windows 7 brings pinch-to-zoom and tap-and-drag control to monitors, overlays and laptops while Snow Leopard supports similar gestures using mice and track-pads.

But users are comfortable with their software working in a certain way – simple, point-click control of pull-down menus which have almost become standardised. It will take something very special to change that. So, while multi-touch technology is clearly suited to tablet computers and smartphones, it remains to be seen if it can find a use in homes and, especially, offices.

Once again, the latest Iphone is a trail-blazing example of “augmented reality”. Point it at a street scene and the built-in compass will overlay heading and directions on the camera’s image. Soon it, and devices like it, will overlay information about the buildings around you, recognise faces in the street and allow us to interact with our environment in ways we haven’t even thought of.

Nintendo’s Wii brought a form of virtual reality to the console gaming market with advanced gesture control, proving that the sheer immersive joy of realistic interaction – such as swinging a virtual tennis racket – is at least as important to the mass market as sumptuous, high-resolution graphics, a victory for function (and fun) over style.

So what is the future for user interfaces? I rather think that there isn’t just one future. Office computers will continue to develop using desk-bound mice and keyboards as hand-held devices and laptops evolve towards gesture-based control, offering innovative ways to interact with their environment.

Multi-touch screens will become common-place in the home, in the hand and for multi-user devices such as Surface if, and only if, the design is good enough to last beyond those “ooh that’s clever” moments and doesn’t, once the novelty has worn off, interfere with functionality.

Of course, we all hope that the future of user interfaces is much closer to science fiction. We want projected holographic images (“Help me Obi-Wan, you’re my only hope”) and virtual-reality headsets (whatever happened to them?). We want computers to react to our eye movements or our thoughts but all this, as well as Tom Cruise’s screen is, for now, still science fiction.

But probably not for long.

Posted in Technology | Tagged , | Comments Off on On the Future of User Interfaces

The Koi Carp Theory of Management

Now that I’m no longer constrained in my corporate straitjacket, I think I can safely publish my “Koi Carp Theory of Management”

No, really. Bear with me.

It came to me in a fish restaurant in Abu Dhabi.

You see, at the time, I had a boss that I didn’t understand. I didn’t understand why he was my boss, why those above him thought he was so great and why, frankly, he even had a job. Nice bloke, don’t get me wrong, but jeez…

Anyway, the fish restaurant in question had huge floor-to-ceiling fish tanks full of fish I couldn’t quite place (note how I avoided the obvious pun?). They were quite big, whitish with red discs and little barbels… ah, they were Koi Carp. I realised that I wasn’t used to seeing Koi from the side, only from the top, usually displaying and looking expensive in the hope I might feed them. For thousands of years, Koi have been bred to look good from above with little interest in what they look like to the other fish in the pond.

And that’s when it came to me: my Boss was exactly the same. He was a Koi.

He must have looked great from above, dutifully doing what was expected of him but, from the side, to his contemporaries, he was nothing special, just an ordinary looking fish.

Of course, to those below him – the scum at the bottom of the pond, enviously looking up at the light – all we could see was an arsehole.

Posted in Rants & Comments | Comments Off on The Koi Carp Theory of Management

Progress…

Ok, the vexed question of what to do next needs an answer.
I DON’T want to:
– Commute
– Work for anyone that annoys me

I Do want to:
– work on the internet again (I miss it)
– write
– work at home

So, nothing for it then… freelance web copy, blogg, e-commerce adviser

Posted in Rants & Comments | Comments Off on Progress…