… and now a few words on closing.

Well, gang, it’s been a lot of fun, and I hope you have gotten something from all this.  Having freed myself from Programmer Hell, otherwise known as the local power company, circumstances require that I shut this operation down.

It will be here for a while, until WordPress decides they need the space, but it will not be possible any longer to moderate your comments.  I expect that the site on Second Life will be shutting down fairly soon as well.  Thanks for visiting.

One more thing.   For those of you who have started following the DW/BI information which had just started, the stock type would not be a separate dimention, but rather a dimensional attribute of the stock dimension.

Gehen Sie mit Gott!

And So, We Start The NASDAQ Data Warehouse

Several years ago I started downloading day prices for Nasdaq.  That is, the open, high, low, close, and volume figures for each trading day for each of the some 3000 or so stocks listed on the Nasdaq exchange.  That’s what is going to be used to build our first data warehouse.  Microsoft SQL Server 2005 has three tools specially created to do this sort of thing, and that is what we are going to use.  Specifically, we will use Analysis Services and Integration Services to do the building, and to some extent we will use Reporting Services for data analysis.

SQL Server Management Studio has some fairly good tutorials on the use of these tools.  However, typical of Microsoft tutorials, you get so buried in the specific details that it becomes very hard to grasp the big picture.  So, before rushing to build something, we need to look at some basics.  The best place to start is with something known as a Data Cube.

The Data Cube Viewed as a Concept

Forgetting for a moment how we actually implement such a thing, let’s just talk about it in the abstract for a moment.  I mentioned in the last post how retail sales is a very popular topic to use in Data Warehouse tutorials.  The ideas there are quite understandable, and when retail concepts are used as the basis for creating a data warehouse, it makes for easier understanding.  Just briefly we will revert to that to make a couple of points, then forge ahead on stock analysis.

Data cubes tend to be oriented towards answering typical questions asked about how the business process is working.  For example, if a company is selling bicycles, a top level manager might want to know how the sales of a particular model breaks down according to geographical location, gender of the purchasor, and age of the purchasor.  In fact, the manager probably wants to know how this works out for each of the models the company sells.  Additionally, since sales according to these factors must surely change over time, then time itself becomes one of the variables to consider.  Let’s consider just the first three factors for a moment, then add the other two.

Now we are going to have two selections of gender, male and female.  Suppose for the sake of argument that we consider three geographical areas where the particular model of bicycle is sold, Northeast (New York, New Jersey, and Connecticut), Midwest (Indiana and Illinois) and Southwest (Texas, New Mexico, and Arizona).  Finally, suppose we have the following age groups of interest: 10 to 19, 20 to 29, 30 to 39, and 40 and above.  We could think of each of these category sets as dimensions, like length, width, and height.  We assume that we have the sales figures for a particular period according to these three dimensions, so imagine that we have a bunch of boxes, 2 genders times 3 geographical areas times 4 age groups or 24 boxes to be precise.  We will label each of the 24 boxes with a particular gender, location, and age group, and place a piece of paper in the box with the corresponding sales figure.  Now we stack the boxes.

We will let gender correspond to height so that the stack is only two boxes high.  Length can be geographical location while width is age group.   The stack of boxes will be 3 boxes long by four boxes wide, by two boxes high, and we now have a data cube.  To know what the sales for this bicycle model is for the time period for which the data is collected, just look for the box with the desired location, age group, and gender.

Okay, the stack of boxes isn’t really what we want to think of as a cube, because its length, width and height are not equal.  Even worse, we are surely going to want to include other dimensions, at least including the other two mentioned above, bicycle model and time period.  Now we have no hope of creating a stack of boxes to illustrate the idea.  Even though we can’t create a real physical stack when the dimensions get greater than three, we can still try to imagine it, and we certainly can implement it as a data cube in a database.

Now, Getting Back to Nasdaq Stocks

That’s enough of selling bicycles, let’s get back to Nasdaq.  In this case we will have at least two dimensions for our cube.  Those will be time and the name of the stock.  We could also further subdivide according to the type of company, such as computer software, disk manufacturer, modem manufacturer, router manufacturer and the like.  Either way we would actually have a two or three dimensional figure which we could actually draw (or stack if we had that many boxes and that much free time).  Now we get specific.

The free day price data is available at eoddata.com. Four weeks of price data is available at any one time, with the prices for all stocks for a given day in a single comma separated value (CSV) file.  It is free to register to download the files, and all you get is the occasional spam ad in your email box.  On the whole, not a particularly bad bargain.

My data set goes back to December, 2005, well before I started working with SQL Server Analysis Services and making data cubes.  I have been downloading the files regularly, archiving them, then putting them in a single table.  Each row has colums for date, stock symbol, open, close, high, low and volume.  The task is to transform that data into a form usable for creating a data cube.  Since the data is already in a database table, we start from there.  Actually we could have started with the CSV files, but since the table is already there that is our starting point.

A Few Words About Database Normalization, and a Link to More Details

One of the first thing you learn about creating a database is that it should be normalized.  There are degrees of normalization, each successive degree having the characteristics of the degrees lower than it, plus an additional characteristic.  I don’t want to get off on that subject too much, so here is a Wikipedia link which discusses the various normal forms in a bit more detail.  The point is that we need a database as a starting point which is not the least bit normalized.  It will contain several dimension tables, and some number of fact tables.  It will be the basis for the initial form of the data warehouse we will build.

A Look Further Onward Before Looking At Details

Much can, and must be said about how to structure this starting, unnormalized database, and that we will do in the next post.  For now, I want to say a word or two about generally what happens during and after the creation of that database.  First we will employ SQL Server Integration Services to populate it with the existing stock data.  Once that is done, SQL Server Analysis Services will be used to create the data cube from it.  Once the data cube is created, SQL Server’s data mining tools will be used to look for trends in the data.  To some extent we will use SQL Server Reporting Services to create some reports on that analysis.  As you will see, Reporting Services does leave something to be desired as a reporting tool, so we will look at some additional approaches for the creation of custom reports.

What’s Coming Next

In our next episode, we will have quite a bit more to say about how to structure this unnormalized starting database, and how to populate it with data using Integration Services.  The episode following that will describe how Analysis Services is used to build the cube.

And Finally, In Closing, a Word About Comments

In a word or two, I’d really like to hear yours.  So far, the comments received on the various topics covered have been most welcome.  With the exception of the spam comments from online poker sites, Russian mail order bride brokers, and the penis enlargement industry caught by the WordPress spam catcher, all the comments received have been most welcome, and have been approved.  With this sudden change in topic, I’d like to know that you folks are still logging on and reading.  See you next time

And now for something completely different …

… to borrow a phrase from the old Monty Python show.  Not that working with OpenSim hasn’t been a lot of fun, but a new opportunity has come along which promises to provide an escape from Programmer Hell, so that is the course I intend to pursue for the near future.  There seem to be quite a few really good blogs on the subject of OpenSim right here on WordPress, so you should find the subject covered pretty well.  Work on OpenSim is not going to be forgotten here, but it is going to have to be put on a somewhat lower priority now.

For the foreseeable future, this space is going to be devoted to implementing databases in websites, together with supporting data warehouses.  There will be a lot to say about the magic buzz abbreviations OLTP and OLAP, what they mean and how they can be made to work together.  We’ll be looking at doing this with ASP.NET and SQL Server, but MySQL and its new companion data warehouse software are going to get a lot of exposure as well.

Those of you who are out scrambling in the job market are likely to be asked the question, “So what process did you use on your last job?”  They want you to say Rational Unified Process, or, God Forbid, The Agile Process (a.k.a Total Chaos).  Generally these, or some other named processes are an excuse not to think.  However, in the data warehouse field, there actually is a truly rational process for getting the job done.  It is known as the Kimball Life Cycle.  It is not the only process favored by the data warehouse crowd, but it is a really organized way to get the job done, and it is going to be mentioned a lot here.

Data warehouses need a lot of data to be useful.  The source for one of the projects discussed here will be day prices for some Nasdaq listed stocks.  Another will be some data gathered online at Second Life.  The stock project will come first.

For those of you not particularly familiar with data warehouses, here is a bit of a preview.  In the ordinary course of doing business, On Line Transaction Processing (OLTP) is the collecting of data as business is being transacted.  It’s the recording of your purchase on Amazon.com, it’s the entry of your account information as you sign up for an account on the local toll road authority, and it’s the buying and selling of virtual land you do on Second Life.  It is the capturing of minute to minute business data during the business day.  On Line Analytic Processing (OLAP) is the process of going back over past recorded data and extracting and understanding trends in the data so as to improve how business is done.

The usual examples provided in OLAP studies involve retail sales.  It is understanding which segment of the customer base favors which product.  It is which sales region moves which product.  Retail sales benefits from this kind of analysis, but so do plenty of other types of activity.

In the case of web based businesses, the OLTP is likely to be done with a database on the web server, as transactions are made.  OLAP on the other hand, is best done offline on a separate database system.  The complete process must include structuring the data warehouse, and setting up the process for regularly extracting, transforming, and loading data from the online database to the offline data warehouse.  SQL Server 2005 has done a pretty good job of integrating the functionality to do this, and it works well with Visual Studio 2005 in creating that functionality.  We’ll see how that works in the first project.

So, to those of you who will be moving on to other blogs, sorry to see you go, but sometime perhaps you will come back for a visit.  For those of you who stay, and the new folks we pick up along the way, here’s hoping you enjoy your stay.

We’ve reached some sort of plateau …

I say that because it’s working in grid mode with two regions, and I could add more with a two minute session on the text editor for another region XML file.  There is a second user, whose avatar has had a sex change operation, and is a man now.  I named the new user in honor of an old friend who grew up with me in the old neighborhood and with whom I am still in contact.

This doesn’t mean that we’re out of the woods yet.  Software problems are bound to appear, and one already has.  What I would like to be able to do this weekend is tell you how I solved it.  Thanks to a very tiring day yesterday cleaning up the last of the broken fences Ike left in my yard, and some things which must get done today, that won’t happen.  Instead, I’ll tell you what I have found so far, and what I’m going to do about it as I get time during the coming week.

I let OpenSim run in grid mode overnight, after having successfully logged on to it, only to find out that the next day logging on wasn’t possible.  A restart solved the problem.  However, letting OpenSim run overnight a second time produced the same problem.  The command prompt where the User Server was running indicated that a login attempt caused a MySQL exception to be thrown.  It was clear that this was something more than an incorrect setting in a configuration file.

Seeing as how there are a lot of people working on OpenSim, and that there is a bug list on the OpenSim site, it seemed reasonable to look there first.  This is the sort of problem that is bound to have shown up already, and probably someone was at least working the problem.  As it happens, someone had worked the problem and a patch was posted.  The symptoms I experienced fit right in with the bug with problem ID 0002099.

A cursory examination of the problem description and the patch seems to indicate that OpenSim makes a connection to MySQL at startup, and expects that connection to be good as long as OpenSim is running.  However, MySQL has a timer on connections, and failure to see continued activity on a connection causes it to time out.  In a situation where people are logging on and logging off regularly, this wouldn’t cause a problem.  However, running a single user for a while, logging off, and letting OpenSim run all night without other logins causes the connection to time out.

That seems to be the nature of the problem, and no doubt the code modification reflected in the posted patch would solve the problem.  So, where do we go from here.  Presumably the preferred approach is to insert the patch, do a rebuild, and go on with life.  That, however, is not what is going to happen.

Now this is the first time for me to use Subversion.  In that part of the software world where people get paychecks and work with Microsoft stuff, Visual SourceSafe is the usual bureaucratic tool to use.  So, the question is, what in the Chicken Fried Hell do I do with this patch?  I assume that somehow, this works with some Subversion functionality so perhaps the first step is to do some reading on this aspect of Subversion.

That particular line of action is not very useful for my purposes anyway.  The idea is not just to get the problem fixed and gone.  The idea is to continue to get a better understanding of the inner workings of OpenSim.  So, the first step is to open up OpenSim with Visual Studio, and look at the two classes which are modified in the patch, and fully understand what is going on.  The assumption is that the change causes a new connection to be made every time a user logs in rather than relying on the one time establishment of a connection to MySQL.  An examination of the code will either verify that assumption or will indicate that some other approach was taken.

Now there is another alternative.  MySQL has a variable, easily set from the command line, which determines the number of seconds MySQL waits before timing out a connection.  Maybe an alternative is just to set that to some incredibly long time.  Perhaps there is a default value one can use which MySQL interprets as never letting a connection time out.  If there are any database DBA’s or security folks reading this, no doubt that last suggestion caused a scream of outrage.  Yes, that is not the best security decision to make.  In principle, the database could be located on some other server on the internet, and this could make it vulnerable to attack by the script kiddies.  In my case, however, the database is on the same server as OpenSim, and that server is not accessible from anywhere except my private network.  In the event that I ever set up an instance of OpenSim on the internet and invite people in, this would not be the best way to solve the problem.

So, no report of progress this weekend, merely an indication of some paths of investigation which will be taken.  I’ll let you know the eventual result, and how it all came out.

With multiple islands and solid objects, we’ve reached a basic plateau …

Things have gone well since the last post.  Multiple regions are running, and Jeannie can’t just walk through objects anymore.  As an aside, I see from other WordPress blogs that she is ordinarily called Ruth in the trade.  As it happens, RL Jeannie has Ruth as her middle name, so it all works out.  As was common with us all in my generation growing up here in the Old South, she was known as a child by first and middle names.  You can just hear her mother now, “Jeannie Ruth, get yo’ ass in this house, gal!”  Ah, childhood!

But, back to business.  Everything was accomplished with the modification or addition of four text files.  Two were *.ini configuration files, and two were *.xml files.  I have posted all four on my website, and have links to each as they are briefly discussed below.

OpenSim.ini was probably the most difficult to get correct.  There are several places in which the storage is declared to be MySQL.  The declarations are of two types.  First there is the one for the dynamic link library containing the functionality to access that particular database type.  Second, there is a connection string for the particular database.  Put the file on a text editor and search “MySQL” to find all the requisite places.  Note, by the way, that you will see the actual connection string, including the password, for the MySQL instance I use.  No security breach there, as the server which is running OpenSim is on a private network not accessible from the internet or wireless monitoring.

Getting the object solid was a two line change in OpenSim.ini.  Look in the “PHYSICS” section of the file.  You will see the choice of a mesher, and a selection of the OpenDynamicsEngine, or ODE, as the physics engine.

There is a second file which relates to connection to MySQL, called mysql_connection.ini
which repeats the connection information for MySQL, once more giving out my secret password.  If you happen to look at the actual URL when you view both of these *.ini files, you will see that the actual file on the website has the file extension *.txt.  This was done to get them to display better on a browser.  Of course, use the *.ini file extension in OpenSim.

Once grid mode is achieved, adding regions is quite painless.  For each region running on OpenSim, there is an XML file in the bin/Regions subdirectory.  OpenSim puts the first one there, and it is called default.xml with the content based on the answers you give the first time OpenSim runs. All that was necessary to add a second region was to add the file region.1000.1001.xml to the Regions subdirectory.  The various XML files can be named anything you like.  I used a convention mentioned on the OpenSim website which identifies it as a region file, and includes the grid position in the name.  The default grid position is x=1000, y=1000, and I took the default initially.  As is pointed out in the OpenSim documentation, these are not meters, just coordinates of slots in the grid.  Hence, this second region is adjacent to the default by being one greater in one coordinate.

There are three things you need to change when you create the second XML file from default.xml.  The first is the region name.  These must be unique within the grid, whether it is your grid or someone else’s grid you decide to connect to.  The second is the listening port.  That first port is, by default 9000.  Each region must have a unique listening port.  I chose 9001, keeping it away from the ones in the 8000 range used by the subordinate servers.

The last thing to change is the region UUID.  There is more than one UUID in the file, but the one for the region, labeled “sim_UUID” is the only one which requires changing.  All regions must have a unique ID, but that only means unique within the grid where they exist.  One way to get a new UUID is to use the GUID tool which is bundled with Visual Studio.  Apparently there is a similar tool in Unix based on a cursory reading of some of the OpenSim website documentation.  For your own grid, with only your regions in it, and you never connect to someone else’s grid.  all you really have to do is change one digit to something else each time you add an XML file for a new region.

So, now what?  I see further progress needs to be made along two fronts.  First of all, this virtual world has to be fleshed out.  More regions must be added, and some extensive terrain editing will be done.  A world of little desert islands is just not enough.  Additionally, a lot of objects are going to have to be built using the basic primitives.  In the process of this, there will be some bugs, some of which may be due to configuration file errors, and some of which will turn out to be code errors.  Those will have to be fixed.

The second area of effort needed, not that the basics are running, is to see if OpenSim is really going anywhere, and if it is do I want to go there too.  There is a lot about OpenSim just on the WordPress blogs, and I need to look at all of it more carefully to see what the rest of the world is doing.  Notwithstanding the occasional frustrations cataloged here previously, playing with OpenSim is great fun.  But, could this be my escape from Programmer Hell?  Is there a shot at making a living with thing while I’m having fun?  Realistically, I rather doubt it.  However, let’s see where this ride takes us.

Houston, we have Grid Mode!

Or, more accurately, we now have grid mode in Houston.

In the last episode of this tale, OpenSim was running on the Vista machine in standalone mode using MySQL as the database.  The hoped for goal this weekend was to have it running in grid mode.  That is just not going to happen.

Knowing Vista’s penchant for looking like it is secure, I knew that I had better be sure to have exceptions in the firewall for the ports used by the various servers.  The port usage is:

grid server – port 8001

user server – port 8002

asset server – port 8003

inventory server – port 804

Now you open a port in Vista’s firewall separately for TCP and UDP.  It was not immediately clear without perusing the code which transport mechanism each server needed, so I just created an opening for both in each port.  Should be ready to go, right?  No.

Each of the user, grid, asset, inventory, and messaging servers seemed to start correctly.  Each was started on a command prompt which was opened as Administrator.  When OpenSim was then started, a dialog from Vista came up saying that opensim.exe was being blocked, and did I want to unblock it.  Of course I said yes, but opensim.exe continued to be blocked.  Error messages showed up on the command prompt indicating that opensim.exe couldn’t access port 8001 or 8002.  So, what more was I supposed to do?

It occurred to me that the problem might be coming from the McAfee Security Center.  That piece of junk was forced on me when I loaded all the software necessary to connect to my employer’s network with VPN.  I work for the local power company, or as I affectionately refer to it, “Programmer Hell”, and we sometimes need to work from remote locations.  hence the VPN.  In the process of loading from the net, the code checks to see if your computer has  acceptable virus protection, acceptable meaning do you have McAfee.  If you don’t, that garbage gets installed.  It seems to have its own firewall, and that has to be turned off as well.

Now to be sure, one of my main goals in life is to find a way to escape from Programmer Hell, and that is going to get done in the fullness of time.  Then I can rip all that crap off the Vista laptop and be shed of it.  For now, however, I’m stuck with it so I’ll have to find a workaround.

Okay, Vista’s firewall and the McAfee firewall are disabled, and it is still no go.  I did a restart as you sometimes need to do with Windows to let it know you really do want the changes, and that didn’t help.  There was only one thing to do, and that was wipe all the OpenSim off the Vista machine, along with the Subversion server.  The short answer is, that under no circumstances can OpenSim and Vista work compatibly.

So, back to the original build which was done on the machine on the network running the small business version of Windows Server 2003.  Based on what I learned about the correct way to set up OpenSim.ini, I modified the Windows 2003 server version of it appropriately.  Then, I made sure that there was a database named opensim on the MySQL instance, and that I had gotten all the tables created previously out if it. (Remember, you don’t have to call it opensim but I did so to stay compatible with the documentation on the OpenSim website.)

Now, to get this build running in standalone mode first.  OpenSim.exe was started, and the SL viewer was started on another computer on the network.  I logged in as Test User, and happily the default avatar, who I have now permanently dubbed Jeannie because she looks like an old girlfriend of that name,  suddenly appeared.  Standalone mode was working.

Now, the big moment.  Will it run in grid mode.  Each of the servers were started in the proper order, then opensim.exe was started.  Remember, by the way, that when OpenSim is running in grid mode, the loginuri command switch you start the SL viewer with is different.  The command line looks like this:

>cd C:\Program Files\SecondLife

>SecondLife -loginuri http://192.168.2.1:8002

Of course, 192.168.2.1 happens to be the IP of the Windows Server 2003 on my network.  Put in what yours is, of course.  The important thing is that you use port 8002 instead of the port 9000 used when OpenSim is in standalone mode.  To make a long story short, everything works fine.  Jeannie is there, with her same old red pants and purple shirt.  She’s standing on that lonely little desert island in the middle of that infinite sea.  But by Damn she’s doing it in grid mode!

So, now what.  For the rest of the weekend, nothing.  There are other things nagging at my time.  But as the days and weeks go on, there is fixing the physics so everything can be solit, getting other avatar shapes so all the avatars don’t look alike, etc.

Now that this contraption will actually work in grid mode, I have to see just how the Chicken Fried Hell you can get a second region running.  Then, I can have two desert islands instead of one.  And, on it goes.  You’ll be hearing a lot about that, and I would like to hear your comments as well.

A preview of the coming weekend …

Greetings visitors.  I see that there has been quite a few of you in the last couple of days, so welcome.  Now that standalone mode seems to be working okay with MySQL, it is time once more to see if grid mode can be started.  That is the big project for the coming weekend.

As mentioned, I may be up against a brick wall ultimately anyway, as far as serious software modification on the existing build.  Here is why.  There are three computers currently in the RL lab.  One runs Windows 2000 Pro, one runs the small business version of Windows 2003 Server, and the laptop runs Vista.  The only way to do any reasonable modification of the existing OpenSim build is to have Visual Studio 2005 and OpenSim on the same computer.  I did do an initial build over the network, with OpenSim on 2003 and VS 2005 on Vista.  That took for ever.  Imagine the glacial speed if I tried to run under the VS 2005 debugger that way.

There are series of problems getting VS 2005 installed on the Windows Server 2003 machine.  The CD Rom drive on it won’t run the VS 2005 installation disks.  Trying to run OpenSim as it currently exists, together with VS 2005 on the Vista laptop won’t work if I try to run under the debugger.  The only way I can get OpenSim to run right is to start it using a command prompt which is started as Administrator.  Unless I can coax Vista to give the debugger Administrator privileges, and perhaps that can be done, the command prompt brought up by the debugger to run OpenSim won’t cut it.  The HTTPListener class instance will not have sufficient rights to listen for HTTP access from the SL viewer.

That leaves moving everything to Windows 2000 Pro, but the Microsoft documentation says that the HTTPListener won’t run on that OS.  That may only be because Windows 2000 doesn’t come with the .NET Framework 2.x, and I have already installed that on the 2K machine.  So, maybe it will, then again maybe it won’t.

The other alternative is just to live with the encredible slowness of trying to run under the debugger with VS 2005 on the Vista laptop, running over the network to OpenSim on Windows 2003 Server.  And it may just mean that I can just live with the software just as it exists and never get clever with my own modifications.

One thing for sure, I am going to give it the old college try in the current configuration (everything on Vista) to at least get grid mode going.  After that, I want to work on some of the minor things mentioned previously.  Then, we’ll see.

Stay tuned, and look for some further results probably Monday some time.

At last, some measure of success.

Finally, I have OpenSim working with an instance of MySQL serving as the database.  I can start OpenSim in standalone mode now and log in over the network with another computer which runs the SecondLife viewer on Windows 2000 Professional.  A couple of things needed to be changed.

As for moving from the default SQLite database to MySQL, it was indeed some bad entries I made in OpenSim.ini.  The message here is just open up the file OpenSim.ini.example in a text editor and save it as OpenSim.ini.  Do this before trying to run OpenSim for the first time.  There are plenty of comments along with the settings, and if followed can be very helpful.  Now, very carefully modify this new OpenSim.ini file you have by reading down through all of the settings, and the commentary which accompanies each setting.  Be careful to comment out all the references to SQLite, and uncommenting and suitably modifying the ones which refer to MySQL.  In other words, don’t get in a hurry.

The original idea apparently was to make OpenSim as flexible as possible.  However, it looks as though all the development effort has focused around MySQL so the flexibility is really not there.  The flexibility comes in part because there are dynamic link libraries for accessing several databases, and it is necessary to reference the MySQL DLL in OpenSim.ini.  You will see the place to do this under a section marked “Storage”, and later under a section which starts with “[Standalone]“.

Be sure that you get the MySQL connection string right, properly referencing the user id and password.  You could use the same root account you got when you installed MySQL.  If you use some other account, you do have to make sure that this account has sufficient privileges to create tables.  OpenSim will do that provided you create a database for it in MySQL and give OpenSim the name of the database you create.  The documentation talks about the database being called “opensim” but you can call it anything you want.  After the first run, all the required tables will be created, and a test user will be added.  The first time you run OpenSim, you will be asked for the test user name, and you could just take the default Test User or give it another name.  The test user will have a female avatar, and hauntingly, it looks like one of my old girlfriends named Jeannie.

In short, carefully following the commentary in OpenSim.ini, and following make the change from SQLite to MySQL and you should be okay.  And by the way, if I haven’t said it emphatically enough, don’t use any database system except MySQL.  Remember, it’s free, and it is the only one with which all the OpenSim features work correctly.  Things work with SQLite, but it is not clear that it will be sufficient when you move to grid mode.

Now there is a brief mention in the OpenSim documentation that if you are running Vista, you probably ought to open a command prompt as Administrator.  There is absolutely no probably to it.  If OpenSim is running on a host which runs Vista, ABSOLUTELY START THE COMMAND PROMPT AS ADMINISTRATOR, THEN START OPENSIM!  Sorry about the shouting, but I just don’t want you to waste an afternoon as I did today getting exceptions when the HTTPListener is started, and wondering why.  This may yet torpedo any development efforts on my part though, since it looks as though I may still not be able to run OpenSim under the debugger.  The Visual Studio debugger creates a command prompt to run OpenSim, but apparently it doesn’t acquire Administrator status, even though I am the computer’s administrator.

If I get time during the coming week, after what no doubt will be another miserable week at the power company, I’ll give grid mode a try and let you know how it comes out.

… and so I start.

I just finished a two week stint without a day off doing my small part of the IT side of getting Houston lighted again after a visit from Ike.  With a little luck, and after getting the last of the storm debris out of the back yard, I hope to get OpenSim running in grid mode finally.  During my previous incarnation as a circuit designer as well as more recently as a software weenie, I’ve always found that I do a better job of getting someone else’s work to work than I have doing original design myself.  We are about to see if that is still true.

Here’s the plan.  Given the assumption that incorrect entries in OpenSim.ini are the cause,  then I need to see just which entries are bad, and why they are bad.  I’ve made the first step, in that I see where contents of OpenSim.ini are stored when OpenSim starts up.  Specifically, the function ReadConfigSettings() is overridden in the class OpenSim, that class being derived from its base class OpenSimBase. This function takes the accessed configuration data and stores it in the base class member functions, which of course are inherited by the derived class.  This is done during bootup.

Now boot up seems to work just fine.  Each of the subordinate servers, User, Grid, Asset, Inventory, and Messaging were all started, following which OpenSim was started.  The problem came during an attempt at login.  A part of the configuration of course is how to get to the database to get user information to determine if the login should be permitted.

The database has been created in the MySQL instance, and there are users in the Users table.  A good first guess is that accessing the table to check the login is not going well.  That could be for a lot of reasons, certainly including that the connection string for the database instance could be no good.  Still, one should not jump to conclusions.  Rather, the plan is to follow the login process step by step in the debugger to see just where the code decides that login  should fail.  That is the big project for the weekend.  I’ll try to report some results before the weekend is over.

Okay, so you want me to do it the hard way …

Ever the optimist, I thought I would give the old build one more try before abandoning it.  I renamed the file OpenSim.ini.example to OpenSim.ini then attempting to judiciously select options necessary to get OpenSim to run in grid mode.  Then, the various servers were once more started: Userserver, Gridserver, Assetserver, Inventoryserver and Messagingserver. Finally, OpenSim was started.  Still, no luck.  An attempt to log in got a failure dialog up which indicates that login failed.

I continue to believe that there is no software error.  The only problem seems to be that OpenSim.ini is not configured correctly.  It appears that there is only one way to determine how it should be configured, and that is to run OpenSim in the Visual Studio Debugger until one sees what is causing the login to fail.  That will take some serious study of the OpenSim code so as to see where to put the breakpoints.  After that, it will be time to try again.  Eventually the mystery of how to configure for grid mode will be opened to us, and you will hear it here.