William (Bill) Vaughn's Musings

September 30, 2005 • Vol.27 Issue 39
Page(s) 23 in print issue

For the past several weeks I’ve been working on the “Getting Started With ADO.NET” chapter of my book. Because there are so many developers out there that have not made the transition to ADO.NET, I think it’s important to get these folks started out on the right foot. Based on the volume of questions in the newsgroups, it’s clear that more and more companies are (finally) migrating to .NET. I’ve stopped lurking on the ADO classic groups as their questions all seem the same and for the most part have already been answered.

It’s comical though to see the students get up on these sites and ask us to do their homework for them. Although I think it’s great that colleges are starting to teach .NET technology along with C# and Visual Basic .NET, it’s sad that so many students think they can get the answer from someone else instead of working it out on their own.

Learning From The Internet

These newsgroup threads also seem to be dredging up the same questions over and over again. Frankly that’s one reason I write my booksso I don’t have to give a short 40-word answer that does not really cover the whole issue. The problem is that some of these developers seem to have learned what they know from the Internet and not from a comprehensive treatment of the subject. I’m convinced that an hour-long session or even a day-long workshop won’t build a foundation solid enough to permit developers to make informed decisions about architectural choices or implementation details.

Although the Internet has a lot of information, it’s often presented in relatively small, jagged chunks that don’t really fit together into a connected picture. That’s what books are formine and the host of others. Don’t get me wrong; I think workshops (like the one I’m doing for VSLive in Orlando Oct. 13) can be a great way for experienced developers to get a better understanding about what’s new in ADO.NET 2.0 and why they should care. I also think that these conferences need to address the needs of those developers just getting startedthose that don’t really know that a garbage collector in .NET is not a union laborer. We can pick out the new folks pretty easily. They are the ones that want to know why the DataReader is not like a Recordset, why they are getting a timeout from the connection pool in their ASP application, why they can’t get connected to an SQL server just inches away, and what’s the best way to pull down 180 million rows into client memory.

A Mentor’s Responsibility

Those that answer questions on the newsgroups really share an important responsibility. As I mentioned in my blog this week (www.betav.com/blog/billva), when we “experts” give advice, we must do so intelligently. Suppose your 5-year-old granddaughter comes to you and asks, “Granddad, where are the matches?” You would immediately discuss with her the dangers of playing with fire. However, if she asked “Where can I find a big bucket?,” you might tell her to look in the garage as you expect she’s big enough to play with a bucket. After all, what harm could she do with a bucket?

The problem is that we often answer similarly important questions on the newsgroups without first asking, “What do you expect to do with those 180 million rows?” or “What are you really trying to accomplish?” When I see a newsgroup question, the answer is sometimes obvious. I have to be careful to balance the short, quick answer (“Just use a DataReader and loop until Read is True.”) with the “right” answer, which is often a question (“Are you nuts?”).

It’s our job as mentors to think about the question. Why is it being asked in the first place? What led that person to the conclusion that what they are doing makes sense? What fundamental problem are they trying to solve? Am I really keeping them safe and helping them stay out of trouble by simply telling them where to find the matches or that big bucket?

No, as some point out to me from time to time, I don’t know all the answers. However, I like to think that I know what I don’t knowor at least I think I do. But knowing the answer is not really enough.

Touring With INETA

As a side note, this week I was pleased to be informed that INETA (International .NET Association, www.ineta.org) has taken me on as a speaker. As a result, I’ll be speaking at .NET user groups all over the country (and possibly the world). My first talks are in Cleveland and Eau Claire, Wis. (I’m planning to watch “Fargo” several more times to get the dialect down before I go.) This also means that your user group can invite me to attend. I really enjoy meeting new people and putting faces and handshakes with names.

Posted by William Vaughn at 07:14 PM | Permalink | Comments (0)

Posted by William Vaughn at 04:46 PM | Permalink | Comments (0)

Posted by William Vaughn at 02:27 AM | Permalink | Comments (0)

Posted by William Vaughn at 07:41 AM | Permalink | Comments (0)

Can't find the File | Export Template menu item in Visual Studio 2005 RC1?

I've been in touch with the Visual Studio developers on campus and we've narrowed down this issue. Apparently, when you first start Visual Studio and choose “Visual Basic Developer” the wizard dutifully disables the wizard’s menu entry. I guess they figure that Visual Basic developers don’t really understand templates and would be confused by the option.

To get this wizard back, simply reload the Visual Basic Settings from the Tools | Options | Import and Export Settings wizard.

Posted by William Vaughn at 03:22 PM | Permalink | Comments (2)

September 16, 2005 • Vol.27 Issue 37
Page(s) 24 in print issue

There’s nothing like the devastation that is caused by a disaster such as Hurricane Katrina to make us all aware how brittle our IT and infrastructure systems are. While I don’t understand how the engineers in charge of the levees failed to recognize the need for pumps that would work during and after a big (really big) storm, I expect that’s something that has taught everyone a lessonalbeit a bit late.

Yes, we all need to be more prepared for the stuff that happens. However, in my humble opinion, we also need to accept that these storms (and the destruction and chaos they cause) are to a great extent self-inflicted. Based on evidence published and validated over and over again by some of the best scientists worldwide, I think it’s all too clear that global warming is directly to blame for higher temperatures in the Gulf of Mexico and the world’s oceans. This, of course, results in more storms and hurricanes like the devastating Katrina, as well as crazy weather all over the planet. In years to come, I expect that we’ll see higher ocean levels and our coastal cities fighting to keep back the water. More of our coastal cities are destined to be fortressed from the sea like New Orleansbelow sea level.

Of course, those with ulterior motives might think and say otherwise. Those who want to continue to depend on fossil fuels and the profits they bring will think and say otherwise. Those politicians whose pockets are lined by the companies in those oil-, coal-, and gas-producing states will say otherwise. Unfortunately, those desperately poor people in Louisiana, Mississippi, and Alabama whose homes and possessions are under water or torn to toothpicks might see this from a different point of view.

We Were Awakened Before

As far as what we can do to prepare for disaster, we in the IT industry know what to door at least we should after we were awakened from our slumber by 9/11. After seeing the bastion of the twin towers and the infrastructure they held collapse into dust in a matter of seconds, most companies decided that more needed to be done to protect themselves and their data. Thanks to the media, I expect we’ll see these buildings crumble in slow-motion replays at every opportunity. The media seems to think that it’s necessary to keep showing us those businesses and people being so horribly lost along with their data and (in some cases) their backup dataas if we can ever forget.

What Can You Do As A Developer?

As IT developers, we share in the responsibility to protect the data and the assets we manage. It’s not just the job of IT management or someone else. Each of us needs to take personal responsibility. Each of us needs to clearly understand that these records we store in databases are often not ours or the company’s; they belong to the people that we call customers. These customers are real people like your spouse, your kids, or your third cousin in Kansas. When this data is lost or compromised, sold, leaked, or simply stolen, the company rarely pays the price. It’s the customer who pays.

When our systems are disabled or destroyed, it’s the customer who can’t draw out his savings to buy medicine, food, and water or to hold off another bank demanding payment. It’s the customer who is trying to get a hospital to treat his injured children but can’t prove that he has insurance or the money to pay. It’s the customer whose identity was stolen and spends years trying to rebuild his good name and credit. These are the real people who are hurt.

Some Aren’t Listening Or Don’t Care

All too often I see Web sites and applications exposed by companies large and small that clearly have not heard and heeded the message. Their sites have not taken the basic steps to prevent SQL injection or otherwise protect their customer’s assets. I would list them here, but that would further compromise their customers’ security. We hear of companies selling or not protecting private and sensitive customer information. If the government wasn’t so busy discounting the science behind global warming and spending countless billions of dollars on war, it might have a few million to spend on making sure that companies large and small protect your and my information and assets.

We as IT developers need to think about these issues when we choose to ignore advice about database security or measures we can take to make sure the data is protected and backed up and that systems are available to fill in when the primary systems are under 19 feet of water.

Another thing we can all do is let the politicians know that we need an energy policy that gets us free from fossil fuels so we aren’t held hostage by the fossil fuel companies. We’ll have an excellent opportunity to do just that in early November.

Posted by William Vaughn at 07:15 PM | Permalink | Comments (0)

Posted by William Vaughn at 07:02 AM | Permalink | Comments (0)

Posted by William Vaughn at 02:07 AM | Permalink | Comments (0)

Posted by William Vaughn at 08:01 PM | Permalink | Comments (0)

Posted by William Vaughn at 05:08 PM | Permalink | Comments (0)

Posted by William Vaughn at 02:44 PM | Permalink | Comments (0)

Posted by William Vaughn at 02:32 PM | Permalink | Comments (0)

In response to a not-that-uncommon usergroup question:

Ah, I'm not a fan of putting BLOBs in the database--despite that it works better now-a-days. Adding BLOBs can easily fill up one of the smaller footprint DBMS systems like MSDE (2gb) and SQL Express (which can hold 4gb). There are a litany of other issues as well:

• Performance. While fetch and retrieval performance has improved, it still takes longer to fetch a BLOB from the database and save it back than it does from a file (even on somewhere on a remote server).
• Code issues: While it's easier than ever to fetch and save BLOBs there are still issues when writing code to deal with these special columns. There are special SQL requirements, DataReader settings and more.
• Cache issues: When you fetch a BLOB, relevant data in the cache can be flushed along with procedures that need to be pushed aside to make space.
• Backup/Restore issues: When you save BLOBs to the database, unless you partition off the segments of the database your backup includes the BLOB data. That might not be a bad idea unless the BLOB data is RO. In this case you needlessly store data that does not need to be backed up past burning the file to a DVD.
• Object use issues: When you fetch a BLOB into a VarBinary or whatever object, you don't have it in a form that Word or any word processor can process. If it's a picture, you don't have it in a form that SnagIt Editor (very cool) can use. Sure, you can bind to a control, but invariably you probably have to write the value to a file first and then point another application at the file.  If the file is RW, once the file is on the client system you have to make sure it's re-persisted to the database somehow. Yes, SharePoint uses SQL Server to store the data in SQL Server. Question: do you plan to replicate this functionality? It's taken them years to get to this point.
• Distribution. When you want to pass updated BLOBs to the client, you must upload the data to the database. In many cases customers have found it far easier to simply distribute the data via removable RO media (a CD or DVD). Some folks taking this approach have encrypted or otherwise protected the data files by adding security headers to the files so common image viewers won’t work.

As far as using multiple databases, I think you're making a lot of work for yourself and painting yourself into a corner. Yes, you can link servers and databases, but the mechanisms are not well understood or widely used. What you're describing is trying to solve the problem of delivering coal with a bicycle by using more bicycles.

I would address the issue of the file format instead of putting the data in a database. I suggest that you can write a security layer to encrypt the files, wrap them in a security blanket that would have to be unlocked with your application--this could be done far easier than by trying to wedge them into a limited capacity SQL Server database.

Posted by William Vaughn at 01:50 PM | Permalink | Comments (5)

While the publisher (Serge Baranovsky) tells me this site is not live yet, it seems to be another good place to extract VB-specific tips, techniques and comments.

Posted by William Vaughn at 11:32 AM | Permalink | Comments (0)

Posted by William Vaughn at 09:05 AM | Permalink | Comments (0)

Posted by William Vaughn at 10:59 AM | Permalink | Comments (6)

Posted by William Vaughn at 05:21 PM | Permalink | Comments (0)

Posted by William Vaughn at 07:18 AM | Permalink | Comments (0)

        Microsoft (to no one’s surprise) is in a bit of a quandary. While the new kids out there in Redmond want to “improve” the now familiar Windows UI (Start button etal.) the experienced ones don’t dare. How many cars out there have “new and different” user interfaces? Sure, the radio buttons and the non-critical components change from time-to-time, but the wheel, turn-signals, wipers and (most importantly) the brake and accelerator pedals are all more-or less the same—standardized all over the world (except for the British that keep them on the wrong side).

        Windows faces the same problem. Since the vast majority of users are now comfortable with the standard Windows user interface it would be foolhardy to change it. My wife (Masters in Math and a smart lady) is not computer illiterate, but almost. She depends on the launch icons to be where they were and everything to be the same each time she starts Windows. When I installed the new version of Office, I slept in the den for a week because the Office UI changed. There are far too many people that just want the computer to work and while everyone wants it to be safer, faster and more stable they don’t want the UI to change.

        If users want anything different they want Vista to be less prone to the filth and garbage polluting the Internet. Today, the Internet is like the middle ages when everyone dumped their waste into the streets where it flowed into a common trench and flowed into the nearest river. Our computers have to pick out the “good stuff” (like our email) out of this river, shake it off and consume it. Users want a new OS that prevents this stuff from making them or their kids sick. Better yet, we all need to stop this pollution from entering the river of data in the first place.  

        As to the “Start” button, for those of you that know how to use a vacuum cleaner, there is only one button: the power switch. It’s also the Off switch too. Should Microsoft rename “Start” to “| O” to accommodate those that don’t now understand that the Start button also permits you to start the logoff/shutdown sequence? I don’t think so. It would just cause more confusion than ever. It would be as popular as the rectangular steering wheels seen briefly in the ‘50s.

Just my $.02.

bv

Posted by William Vaughn at 09:08 AM | Permalink | Comments (2)

We’ve had another newsgroup developer ask if it makes more sense to convert an existing Visual Basic 6.0 application to Visual Basic .NET or start over “from scratch”. We’re going to get this question more and more over the next decade (or two) as people gradually move to .NET. I think it would be useful to discuss the issues involved. No, I don’t know all the answers as my focus has been on data access interfaces.

As far as the data access layers of your application, I can see several alternatives. None are completely satisfactory as each has their own set of issues.

• Run existing COM-based ADO (what I call ADO “classic” or ADOc) code in a .NET application. The Visual Studio conversion wizards can move about 90% of your ADOc code so it can run in a .NET application using the COM interop layer—assuming you use Visual Basic .NET. COM interop puts a wrapper around the COM classes and tries to translate what COM expects to what .NET expects and back. Unfortunately, ADOc does a number of things that COM interop will never understand. This means that you’ll find that some code simply does not work the same. Sure, most of it will work, but you’ll forever be wondering if that issue you’re fighting is caused by COM interop or something you’ve done wrong.
• Convert existing ADOc code to ADO.NET (even version 2.0) can also be problematic. The default Recordset in ADOc is a server-side cursor—ADO.NET does not now and will not support server-side cursors for the foreseeable future. The earliest we can expect to see any server-side implementation is in Orcas (some time in the next three years). If your application is not built around server-side cursors (as most ASP applications) but uses ADOc disconnected Recordsets, the logical conversion from and ADOc function to an equivalent ADO.NET function is pretty easy. What makes things harder is that while the class names in ADOc are similar to the ADO.NET implementations, they are really very different behind the class names. I like to say the only thing ADOc and ADO.NET share in common is the fonts—but it’s not really that bad. Given the differences in functionality and features, I don’t think it’s useful to try to convert ADOc to ADO.NET on a line-by-line basis. Instead, I would examine a data access component and determine how to implement the equivalent functionality in ADO.NET—now that can be fairly straightforward. This means if you have a function that executes a stored procedure or ad hoc SQL query and returns a rowset, you can write that very quickly in ADO.NET. Can Visual Studio help with the conversion? No, not really but it can generate code for you if the function is fairly simple.
• If you’ve built a multi-tier application then you might consider converting the application in phases. The data access classes might be left in ADOc at first but in any case you’re going to want rethink this strategy eventually.

    The real disadvantage to ADOc is its dependency on the MDAC stack. Fortunately, Microsoft has recently addressed that issue as well. The new SQL Server Native Client OLE DB provider can be used in existing ADOc application—but only when accessing SQL Server. The new SQL Native Client provider is not dependant on MDAC. This means you'll be able to access all of the SQL Server 2005 features from VB6 (although I have not tested this...).

    If you’re still using JET with your VB6 application, you’re stuck with OLE DB in ADO.NET as well. While ADO.NET applications are not as susceptible to DLL hell issues, you’re still coupled to a deprecated technology—unless Microsoft Office decides to use the shock paddles on MDAC. I’ve seen some towers being set up on campus to collect lighting so who knows. As you convert to .NET I suggest you consider switching away from the home-office JET database to the better-supported and more secure Express version of SQL Server.

    When we build Visual Basic 6.0 data access applications, we usually leverage the built-in (or more likely) third-party bound controls to construct the UI. When you migrate to Visual Basic .NET you’ll discover that the language is vaguely familiar but the UI controls are not—especially when you try to use more sophisticated controls like the DataGridView or other complex bound controls. The properties, methods and methodologies are very different. The new controls have a bevy of features that make it easy to do some things you wrote code to support in VB6. They're also missing some properties and methods so plan on rethinking your approach to bound controls. Data binding in the CLR languages has been totally rewritten (and rewritten again in the 2.0 Framework). That’s the good news. The bad news is that this means the binding code and techniques you used in Visual Basic 6.0 won’t work—or at least not in ways you would expect. The familiar cursor-based ADOc ISAM Recordset is gone. It’s been replaced (sort of) by the DataTable. You can’t “MoveNext” in a DataTable—you don’t need to. I could go on (and I do in my books on ADO.NET), but you get the idea. The best thing to do IMHO is simply forget ADOc and Visual Basic 6.0 and consider the .NET platform as something new, more powerful and something you need to learn. Those that don’t can plan on early retirement.  

    I encourage comments on this process. Have you had a good, bad, successful, unsuccessful, easy, painful, never-ending experience converting from Visual Basic 6.0? Post your comments here.

hth

Posted by William Vaughn at 01:26 PM | Permalink | Comments (0)

Posted by William Vaughn at 11:36 AM | Permalink | Comments (0)

September 2, 2005 • Vol.27 Issue 35
Page(s) 23 in print issue

I'm still heads-down working on the next edition of the "Hitchhiker's Guide to Visual Studio and SQL Server 2005." It's slow going, and it's been hot here in Redmond. My office is like an ovenover 90 degrees. When my P4 3.2GHz system's processor reached 171 degrees, I decided to install a liquid-cooled CPU heatsink before solder started dripping off the motherboard. The new cooling system, which includes a pump, fish-tank hoses, and an external heat exchanger, dropped the CPU to a balmy 136 degrees. It’s also kind of sexy. The pump and heat-block glow an iridescent green. All I need now are those clear plastic cabinet covers, but I digress. . . .

Cramming Valid Values Into A Parameter

Last week I managed to complete the Building Commands chapter that I talked about in my last column. As I worked though the section on building a Parameters collection, I stumbled over a problem that has been there since the first version of the .NET Framework.

The problem has to do with how (and when) the Parameter object's Value property is validated. In ADO classic when we tried to set the Parameter Value property with a value that was either too long or not the right type (for example, a string instead of a number), ADO threw an error immediately. This made the error handling a bit tough as you had to wrap the code that set the Parameter with an error handlerone for each Parameterbut at least you knew which column was in error. On the other hand, ADO.NET does not care what you put in the Value property. You could stick your granny’s nightie in there (if you could make it an object) and ADO.NET would not care. However, when you try to execute the Command, ADO.NET throws an exception that says it can’t convert, that there would be string truncation or some such, but what Parameter is it talking about? This fact it keeps to itself.

Validating Via The CLR

So, what should you do about this? Well, my co-author Peter Blackburn and I worked for several days coming up with a solution. We tried several (some rather complex) approaches. When you construct a Parameter object you can set the intended datatype and the size (if it’s a variable-length type like a VarChar), and these settings are exposed as properties. However, it’s not that easy to compare a system type with a provider-specific type such as a SqlType. I started by building a giant Case statement that made the comparison and ran a whole set of values through it. It worked, but it was like screwing a 4-ton air conditioner on the back of my computer cabinet to solve my heat problem. We needed a better solution, and we finally hit on something simple.

We created another variable of the intended type and set it to the intended value. This was done in a Try/Catch so it would fail if the variable was not the right type. Testing for length was far easier. It’s funny how this solution seems so obvious once you figure it out.

Validating At The Source

The real question is when should this test be made? Over the years I’ve learned that data validation is absolutely critical to data processing. I can’t count how many times a simple data overflow or type-check issue has brought an entire system to its knees. But data validation problems are best solved early in the processwhen the data is first created. If users are involved, you must consider that they might be the authority on the values. They provide the eyes to read the form or credit card number they’re transcribing. Only they know the birth date and the invoice or part number. Your application might (and should) provide lists of these values when they are known so the user simply has to choose from a date picker or a drop-down list. But when this is not possible, it’s up to you to validate this input dataand doing so just before you stick the value into a Command Parameter value is too late.

Validation needs to be done at the sourcelike in the TextBox control as the data is entered. This might mean creating a custom TextBox control that validates the keystrokes as they come in. This is not hard to do (we include a sample in the book), but it ensures that a numeric field only has numbers and a decimal pointonly if the numeric datatype is not an integer.

CLR languages such as Visual Basic .NET and even C# make it easy to create custom controls to validate data. There are a wealth of these available on the Internet, and because they provide the source, they’re easy to use and tune. I suggest you try building a few of these the next time you have to make sure that the object you’re trying to stick into that Command Parameter fitswithout worrying if the sleeves get cut off.

Posted by William Vaughn at 07:27 PM | Permalink | Comments (0)

Posted by William Vaughn at 10:08 AM | Permalink | Comments (0)

Posted by William Vaughn at 06:52 PM | Permalink | Comments (0)

Posted by William Vaughn at 05:57 PM | Permalink | Comments (0)

There is nothing like a disaster like the devastation caused by Katrina to make us all aware how brittle our IT and infrastructure systems are. While I don’t understand how the engineers in charge of the levees failed to recognize the need for pumps that would work during and after a big (really big) storm, I expect that’s something that has taught everyone a lesson—albeit a bit late. Yes, we all need to be more prepared for the stuff that happens. However, IMHO we also need to accept that these storms, and the destruction and chaos they cause are to a great extent self-inflicted. Based on evidence published and validated over and over again by some of the best scientists world-wide, I think it’s all too clear that global warming is directly to blame for higher temperatures in the Gulf of Mexico and the world’s oceans. In the short term this means more storms and hurricanes like Katrina and crazy weather all over the planet. NPR just reported that the forecasters have bumped up the number of expected storms--they expect far more storms and far more large, dangerous storms. We aren't out of the woods yet.

Thanks to our inability to curb our appetites for fossil fuels, in years to come I (and the majority of scientists who should know more than I) expect that we’ll see significantly higher ocean levels and our coastal cities fighting to keep back the water. This means more of our cities are destined to be fortressed from the sea like New Orleans–below sea level--cities like Washington D.C., New York, Houston, Miami, Los Angeles and Seattle. Of course, those with ulterior motives might think and say otherwise. Those who want to continue to depend on fossil fuels and the profits they reap will think and say otherwise. Those politicians whose pockets are lined by the companies in those oil, coal and gas-producing states will say otherwise. But this is not a Republican, Democrat or Fascist issue--all of the administrations since our first warning in the Carter years have caved into the energy lobbies. As a result, the new energy bill only gives lip service to replacing fossil fuels to power our economy. It’s another windfall for the Saudi’s, the oil companies and George Bush’s friends in Texas. It won't be long before we'll think that $5.50/gallon gas is cheap--the people in Europe do. They have paid over $5.00/gallon for some time now. Unfortunately, the desperately poor people in Louisiana, Mississippi and Alabama whose homes and possessions are under water or torn to toothpicks might see this from a different point of view—if they live through this.

I think we should launch a NASA-scale program to develop solar, wind and other alternative forms of energy. We need to figure out how to get salmon over and around our hydro-electric dams and cut regulations that prevent wind farms from being built. This money will be spent here in the US hiring American engineers and use American factories and technology. Just as NASA fueled the integrated circuit industry (that spawned the personal computer), this new technology will free us from dependency on foreign sources of energy.

You might wonder why I would use the “F” word to refer to a government. Some see fascism as a government where the corporations elect representatives for the Congress and Senate--“corporatism”. They think that what's good for the corporations is best for the country. This is not a new idea. Given the lack of restrictions on campaign financing and that many politicians spend much of their time raising money to win their next election; you can see why I would come to this conclusion.

Posted by William Vaughn at 03:44 PM | Permalink | Comments (4)