History of Mindat
From Mindat
| Table of contents |
About this History
This History of Mindat document was originally prepared for the 10th Anniversary of the Mindat project (December 25th 2003), but has since been extended, and has become part of the material used in the talk given by Jolyon Ralph to Mineral Collecting Clubs about Mindat.org
A Brief History of Mindat Time
25th December 1993
I start development of a DOS program. The DOS program is designed for my own use, as a simple text-entry program for retrieving data on minerals and mineral localities. The data entry is done manually by editing text files, one file for each letter of the alphabet containing minerals starting with that letter.
I started the work on Christmas Day 1993 because I was bored - there was nothing good on television!
I originally planned on writing a database using the Paradox database system, but I could not find a way to format out chemical formula correctly, and I was adamant that I wanted something that displayed formula correctly, not in a simplified-for computer way, such as CaCO3.
So I fired up Borland C++ compiler, and started to write my own program from scratch.
28th August 1995
Final changes made to the DOS program. You can view the source code here (http://www.mindat.org/mindat-source.php) , and an example data file from the time here (http://www.mindat.org/mindat-old-e.php). By this point, the program had evolved so that data could be added in directly from the program, but it was still text driven and hardly user friendly.
Late 1995
Having started using Windows 95, I decided it would be a good exercise in learning Windows C++ programming and the Microsoft MFC framework to convert my old DOS program to a Windows application. I spent a lot of time trying out the new controls and methods of Windows 95 programming to try to learn how to use different techniques of Windows programming. To date it's still the most complex program I've ever built on Windows, and most of the skills I learnt doing this have not really been used again.
The mindat logo was born at this time too!
The mindat logo was a Siberian Elbaite, taken from a 19th century hand-coloured print (long out of copyright).
The search pages were quite powerful in mindat32 - an example of how it looked is shown to the left:
The locality search gave a hierarchical list that could be clicked to open and expand entries.
Mindat32 used the same database files and format as the original mindat DOS application, and that was beginning to reveal major weaknesses.
October 1996
By October 1996, I was looking at alternative methods of displaying mineral data and fixing one of the big problems with the system - it was difficult to format things nicely for the page, and difficult to keep that formatting the same when printing, and of course, dealing with international differences in paper sizes, etc, was frustrating. Around this time, Microsoft made available Internet Explorer 3.0, and a programmable interface for allowing use of Internet Explorer 3 technology within your own applications was provided. And because at that time everything seemed to be ended up being called 'something' Explorer, mindat32 was rebranded as Mineral Explorer '97 (which was a terrible name). The design work improved, and the buttons and toolbars looked a lot nicer.
The other major advantage was the database system was improved dramatically for performance, however this lead to problems with Windows 95 users - the performance gain was generated by splitting the data into individual files for each mineral, and locality information with minerals was 'normalized' by storing locality ID numbers rather than locality names.
Mineral Explorer hit a natural stop in development - by this time I was already distributing copies of the software from my website, and Bob Keller's Rockshop was providing download links for the software. The big problem was how to deal with updates to the data, I did not have time to keep adding all the localities myself, and there was no simple solution for merging in updates sent by users into the master database - however this was the solution I was working on.
However, development of Mineral Explorer slowed down. In Feb 1997 the software development company I had run with my parents since 1990 closed down and I had to look for other work. Mineral Explorer proved a great demonstration of my skills, and I was offered work with several companies, however in the end I took a job for a local company doing Windows and Networking consultancy rather than programming, and for three years I did very little programming work, and Mineral Explorer was put on the shelf as I concentrated on the serious matters of my new career (and for the first time in a long while, girlfriends).
Late 1999
At the end of 1999 I got very frustrated with my job, in particular I had the 'creative itch', I wanted to go and make things, get back to programming, rather than just go out and be a (highly paid) problem fixer for companies computer problems. I took a gamble and quit my job in the start of 2000 to start my own company to do internet development and programming, called Mysterious Ways Developments Ltd (http://www.mways.co.uk).
Late 2000
Towards the end of the summer I was experimenting with PHP (http://www.php.net) and MySQL (http://www.mysql.com), two new technologies I had not used before.
I had successfully taught myself C programming on DOS with Mindat, and C++ programming on Windows with Mindat32 - so it made sense to throw myself in the deep end again and try to develop something enormously complicated as my first real attempt at PHP web programming.
I ditched the awful Mineral Explorer name and went back to the Mindat name, and registered mindat.org as a domain name, and after a surprisingly short time, the first version of mindat.org was ready in Late September 2000.
10th October 2000
I announced mindat.org on the Usenet news channel sci.geo.mineralogy (http://groups.google.co.uk/groups?hl=en&lr=&safe=off&group=sci.geo.mineralogy) with the following message:
From: Jolyon Ralph Subject: www.mindat.org Newsgroups: sci.geo.mineralogy Date: 2000-10-10 02:44:30 PST I need a few people to come along and test my new website, an online mineral reference site - it's called www.mindat.org, it's still under construction and so a lot of pages don't work, but have a look and tell me what you think! Regards, Jolyon
My first reply wasn't encouraging!
From: Jacques Jedwab Subject: Re: www.mindat.org Newsgroups: sci.geo.mineralogy Date: 2000-10-10 05:30:15 PST A real mess with chemical formula typography. Long way ahead to go for you. YOU need to have a look at other mineral data banks. (see Athena, Barthelmy and their links). I hate to tell you that you are going to waste your time. Try something more specific or advanced, like mineralogy of meteorites, oceans, slags, deserts, painter pigments; old instrumental crystallography; history of UK mineralogy; Cornish & Scottish miners during the California Gold Rush; recognizable minerals used by alchemists; prehistorical pigment mineralogy; industrial minerals, etc., etc. Regards, J.J.
Luckily I persevered! And to be fair to Jacques, he did reply a few days later
From: Jacques Jedwab (jjedwab@ulb.ac.be) Subject: Re: www.mindat.org Newsgroups: sci.geo.mineralogy Date: 2000-10-12 00:18:24 PST After running only a few days (and receiving many criticisms), one must conclude that numerous people were awaiting such a data bank, which fills a gap... Good luck to jralph! J.J.
Early 2001
By 2001 the site was active and popular, and I was lucky enough to get permission from John Betts from www.johnbetts_fineminerals.com to reuse his photographs. But course, the most important innovation was the solution to the data entry problems, by becoming a website rather than a software application, the problems with distributed data entry and validation had been solved.
25th December 2003
Mindat.org has grown steadily and evolved smoothly since October 2000, and by the 10th anniversary of the mindat project we now had by far the largest public mineralogical database on the internet, and according to my statistics, almost certainly the busiest website related to minerals or mineralogy in the world.
February 2004
The server fails! The 1.7Ghz processor machine, installed less than 18 months previously, has died. At short notice, we rush out to buy a replacement computer (2.5Ghz, 1Gb RAM), and mindat.org is up and running again a short time later.
Late 2004
As we came up for our 11th anniversary, mindat.org was regularly receiving 2 million page views per month, distributing an enormous amount of traffic. The upgraded server is starting to strain, and people are continually seeing connection errors and slow response as it struggles to keep up. Mindat.org starts a simple advertising banner campaign to help raise funds for a more powerful server and hosting to replace the current system.
14th March 2005
Disaster! The mindat server fails (less than 13 months after it was purchased, and conveniently 1 month outside of warranty period). Plans to purchase a serious replacement are brought forward and a temporary read-only server is constructed out of an old system. Double disaster! The backup procedures hadn't been working properly for nearly a month, backup archives were corrupt and meaningless, and all data since 17th February 2005 has been lost. The first serious data loss in the history of mindat (not something to be especially proud of).
The messageboard suffers worse, only messages from 2004 were backed up. We were able to retrieve about 30% of missing content by copying data from Google's caches of our old pages, but not everything was available to us.
The primary hard disk was sent to a data recovery firm (98% success rate), however our drive was in the 2% that they couldn't help with. It was not a good week.
21st March 2005
