An Update on the Updates to SCAtoday.net

The first day was a little rocky, but hopefully things are better now. This article explains a little about the upgrade, why we did it, and what's planned for the future. The article touches on technical points, but I'll try to keep it from getting too dull.

A Little Background

Or, "Justin! Why did you change everything?!"

For small web sites such as personal home pages, creating the pages in hypertext markup language (HTML) is fairly simple. That won't work for large sites with a lot of pages, especially when (as with SCAtoday.net and other news sites) there are a lot of changes to the site every day. For that reason, most large web sites use software called a content management system (CMS) to create the pages. With a CMS, editors type the pages (or, in the case of a news site, the stories) into a web form, and the software stores the pages in a database. The CMS retrieves articles from the database to display listings (such as our home page) or individual articles.

SCAtoday.net originally used a custom-built CMS called "Today", written by me, Justin. "Today" was modeled after the original version of a site called http://linuxtoday.com/ for which I used to be one of the code developers. "Today" was completely original code, all my own work, but the general layout of the site was designed to look like LinuxToday used to look (LT is now much different in layout, with more ads and such...frankly, I don't like it as well).

The original plan for SCAtoday.net was, quite frankly, to have fun writing the code. The fact that we would provide a news site for the SCA and other living history groups was incidental, just an excuse for me to take on a programming project that sounded like fun. Milica, my wife and now our Managing News Editor agreed to help out with editing. We saw a need for a site that was updated daily, since most of the existing sites focused more on a "magazine" format, with updates monthly or quarterly. Instead of doing feature-length articles every month, we wanted to do very brief news items every day. Originally, the goal was to have one news item every day.

Three things have happened since SCAtoday.net debuted, and all three of them were a little bit of a surprise to us:

  1. Milica found that she really enjoys editing the news site, and she's absolutely taken the proverbial ball and run with it. We now average over three articles per day, and about two-thirds of our stories are Milica's contribution.
  2. The idea of a daily-updated news site took off in a way that we never dreamed it would, and our readership has exceeded all expectations.
  3. I got really busy with my modern-world job, with several SCA offices (including being War Chirurgeon for Pennsic last year), and with moving into a new house. Milica has somehow managed to stay on top of the editorial workload, but I was falling desperately behind in my work on the "Today" CMS software.

Resignation and Conceding Defeat

About six months ago, I finally admitted to myself that "Today" just wasn't going to ever get finished. I'm really proud of the quality of the software that I did write; it was reliable, fast, and easy for our editors to use. The problem was that as our readership grew, there was an increasing demand for more features such as forums, link directory, and an event calendar. All of these things needed, as a foundation, a hierarchy of categories (basically, a subject index such as you would find in a library catalog). It may not seem like much, but writing software to do this is a very complex undertaking, if you are going to do it right.

I had a completed design for the category manager, but no time to write the code. I didn't even have time to add an event calendar — a much-requested but complex-to-program feature. Reluctantly, I admitted that I needed to start searching for an off-the-shelf content management system. "Today" was good, but was never going to be completed.

The Search

Content management systems (CMSs) are to programmers the way sports cars are to serious motorists. You don't just want something to get from point A to point B — you want something that has the feel of fine engineering, something that will make you enjoy the ride. Having written my own CMS which, although incomplete, was exactly what I wanted it to be, I was very finicky about the replacement for "Today".

There are a lot of CMS packages available, both as commercial software and as free software. Since SCAtoday.net is non-profit and funded out of pocket by Milica and me, the purchase of expensive commercial software was out of the question. Also, our web server runs Linux, and Linux software is what I use professionally, so it made sense to choose an Open Source free software program. That would also allow me to modify and tweak it as needed, something not possible with most commercial software. Finally, I really preferred that the software be written in the PHP programming language, because that is well-supported by our server and also is what I used to write "Today" (so the data conversion would be easier).

I looked at PHP-Nuke, PostNuke, MoveableType, and several other CMS programs, but nothing had exactly the features I wanted. Finally, by accident, I stumbled upon a package called Drupal and decided to take a look.

Drupal is written in PHP, works well with the MySQL database that we use, and is freeware. What initially attracted me to it, though, was its handling of category hierarchies. Not only can it have a subject catalog with nested sub-categories, but it can have multiple such catalogs that are all independent of one another. The concept of taxonomy in Drupal is generalized, allowing catalog trees to be much more than just subjects. You can, in fact, create nested category trees for many other purposes.

Drupal is also modular, which means that you can add on features to the basic program easily. There are existing modules for event calendars, "Yahoo!"-like link directories, browsing news archives, and many other features, or you can write your own.

There are a lot of good CMS programs, each with its own pluses and minuses, but Drupal seemed to be the best fit for SCAtoday.net.

Upgrading the Site

I started working with Drupal about four weeks ago, and working with it seriously about three weeks ago. The most difficult part of the project was figuring out how to move the data from Today to Drupal. They both use the MySQL database, but the layout of the tables is totally different. Finally, I ended up writing a custom program in PHP to do the data migration. This program took most of a day to write, was used once, and will never be used again now that its work is done.

At 3:00 a.m. on Monday morning, I had finished the data migration and was ready to make the new SCAtoday.net "live" on the web server.

So What Happened to the Fonts?!

Oops! This problem is proof that there is no such thing as "enough testing." I've been a web developer for a long time, long enough to know that people have a huge variety of different browsers on their systems. So I test new web sites with four different browsers: Konqueror (part of the Linux KDE desktop environment), Netscape Navigator, Mozilla, and Internet Exploder, err, Explorer. SCAtoday.net worked fine with all of these in my test system, and I tweaked the layout until it was as close to the old site layout as I could get.

What I didn't realize is that all four browsers had settings that said, "No matter what the web site wants, never ever let a font get below a certain minimum size." So my test browsers &emdash; all four of them &emdash; failed to show just how tiny the SCAtoday.net site's new fonts really were. It looked fine on my system, honest!

A couple of readers pointed out the problem on Monday night. I'm amazed that I didn't get dozens of emails right away, but SCAtoday.net readers must be a very kind and patient group. I immediately changed the stylesheets which determine font sizes. Things aren't perfect yet (the event calendar still has some weirdness), but it is hopefully much better than it was. If your browser still shows the fonts too tiny, and other web sites look okay to you, please email webmaster@scatoday.net with details of your browser type so we can troubleshoot the problem.

Looking Toward the Future

The forums are installed and ready to use, but we haven't really publicized them yet. If you have an idea for a forum that you'd like to see, send an email to forums@scatoday.net, which will reach Catriona nicHugh Mclaey, our Forums Editor.

The event calendar is ready to go. Please feel free to start contributing event notices, and we'll get them published. You will need to be a registered user to submit event notices; we established that policy because we may need to contact someone if we have questions about the event, and we have no way to contact anonymous contributors.

There are more things you can do with a registered user account now than ever before. Drupal actually lets users have control over some of the features on the home page, so you can turn things on and off to your own liking. We don't have automated login working yet, but it's being worked on and we hope to have it up and running soon.

We have the links directory up and running, but right now there's not much in it. That will soon change; among other things, we plan to put in all of Aoife's links that she has kindly contributed to the world. The links directory will contain, in general, links to web sites rather than links to specific ephemeral news items.

As you may have noticed, items on the newswire now are cataloged by topic, by news source (if applicable), and by article type (humor, opinion, etc.). All the newly-posted content has these topic links, but most of the older articles do not. We will be working on that over time, and eventually all the articles on the site will be cataloged.

Drupal has a very nice built-in search engine. I don't have it enabled yet on SCAtoday.net because it needs to run a really long batch job to index all the existing archives. It's coming Real Soon Now.

There are some other annoyances. I've been told that people have trouble finding the link to "click through" to original stories on other web sites; I'm working on that as a high priority. We also don't have the automatic paragraph marks from blank lines in content any more, but again, that's being added. Please be patient, and we will work on the issues as fast as we can. If you see something you'd like fixed, send a polite letter to webmaster@scatoday.net and I'll add it to the to-do list, if it seems feasible.

In the near future, we will be offering registered users the option of having newsfeeds from other sites on our home page. Suggestions of specific sites that you would like us to include in our list of available choices should be sent to webmaster@scatoday.net for consideration (they need to be on-topic, of course).

As always, thank you for visiting SCAtoday.net, and thank you for your patience during this time of software-induced turbulence. We hope you'll find the many new features worth the short-term aggravation.

—Justin