Cold Black Coffee

Visual Studio 2008 Renaming Bug

2010-06-05T13:49:00.005-07:00

I'm a big fan of the renaming feature in visual studio. You can right click on any variable in your code and select Refactor -> Rename. Then rename the variable once and all other references are updated.

Except I like to inject code right into my .aspx pages instead of the code behind files sometimes. The syntax for that is:

<% Response.Write(SOME C# CODE); %>

The rename functionality won't find these references in the .aspx files. Instead I get a compile error and can then rename the remaining references easily enough.

It seems like this should be fixable. The fix might hurt the performance of the renaming feature a little bit but I think it would be worth it. Does anyone know if this is already fixed in VS 2010?

SSL Hell - Multiple web servers in a cluster

2010-06-05T11:32:00.004-07:00

I was following the instructions that GoDaddy provided to install an new SSL cert on IIS6. These instructions cover how to get up and running on one web server, but they don't detail how to export the cert to other web servers. The site I was working on had several in a cluster.

So I thought I'd detail what worked for me. The trick is to export the cert to a .pfx file. Note: the steps below are intended to be followed after you've already set up the certificate on the first web server.

To do this, first start with the certifcates snap-in all set up. GoDaddy's instructions for this are pasted below:

Click the Start menu and click Run.... Type mmc in the Run window and click OK to start the Microsoft Management Console (MMC).
In the Management Console, select File then Add/Remove Snap In.
In the Add or Remove Snap-ins dialog, click the Add button and then select Certificates.
Choose Computer Account then click Next.
Choose Local Computer, then click Finish.
Close the Add or Remove Snap-ins dialog and click OK to return to the main MMC window.

Now you're ready to export:

Browse to Personal -> Certificates.
Right click the certificate you need to export in the right hand window and select "All Tasks" -> Export ...
Click Next on the first meaningless page of the wizard
Select "Yes, export the private key" and click Next
Leave the default selected (just "enable strong protection" checked) and click Next
type in some password that you won't forget and click Next
Browse to a location to export and click Next
Click Finish

It is important that you do specify a password for everything to go smoothly. Now for the importing:

Copy the .pfx file to an additional web server
get the certificates snap-in up on the new server (same as instructions above)
Under Personal -> Certificates right click the folder and select "All Tasks" -> Import ...
browse to the pfx file (will need to change the "File of Type" drop down in order to see it)
click Next through the wizard, leaving the other defaults.
Don't forget to import the intermediate certificate! That is the same as the first web server.

That is it. Now in IIS you can go specify that cert. You won't be responding to a CSR (certificate signing request) like the first time. You'll just be choosing an existing certificate.

Query is fast in management studio but hangs from Console App

2010-03-16T14:19:00.005-07:00

I'm going to include my little narrative of how I troubleshot this issue. But first let me just get the answer out there. You need to include this at the start of your stored procedure:

set arithabort on

My troubleshooting story is also pretty concise I guess. I used SQL Server Profiler to see what my app was doing differently from management studio. The actual call was identical, but any .NET MSSQL connection calls a bunch of set commands before it does anything. The entire list I found is below:

set quoted_identifier on
set arithabort off
set numeric_roundabort off
set ansi_warnings on
set ansi_padding on
set ansi_nulls on
set concat_null_yields_null on
set cursor_close_on_commit off
set implicit_transactions off
set language us_english
set dateformat mdy
set datefirst 7
set transaction isolation level read committed

By starting with a fresh query editor window and adding in these set commands one at a time I found the culprit pretty quickly. Then I just set the property the way I want it inside the proc to override the damage the connection from .NET causes.

StackOverflow was also really helpful in two posts. One focusing on the SqlDataAdapter and another that was more generally about SqlCommand. Just go help vote up the answer that this post is consistent with. Unless you find otherwise, then let me know.

What would be nice is being able to control which set commands are issued from my app. I've found nothing on how to do this. If you find something please share!

Automated Speed Monitoring with MSSQL

2010-03-02T21:10:00.014-07:00

What? Why?

I'm in the web data extraction business. That means I'm often writing hundreds of thousands of rows to a table over the course of days. Well when you're dependent on a third party website for your data, a lot can go wrong over the course of days. So I started formulating my own table monitoring solutions.

At first I would just periodically do a SELECT * here and a SELECT count(*) there to get by. But when a weekend goes by and on Monday morning you're 10K rows short, you want to know if things just slowed down a little or if everything broke sometime Sunday night. My solution is to automate my little SELECTs to happen on a regular schedule and then store themselves in a log table. By comparing the latest row as I insert I can do an on the fly rate calculation as well. Sound useful? The details follow ...

How.

First I needed a log table. I made something like:

CREATE TABLE [dbo].[SpeedLog](
[Count] [int] NULL,
[Timestamp] [datetime] NULL,
[PerHour] [int] NULL,
[PerDay] [int] NULL
) ON [PRIMARY]

Next I made a view that produces a new row to insert into this table. I find it handy to have this logic isolated in its own view rather than having an enormous insert statement,

CREATE VIEW [dbo].[vw_Status]
as
SELECT
count(*) as [Count]
, CURRENT_TIMESTAMP as [Timestamp]
, (count(*)-MAX(lRow.UrlSnapShotsCount))/DATEDIFF(hour, MAX(lRow.[Timestamp]), CURRENT_TIMESTAMP) [PerHour]
, (count(*)-MAX(lRow.UrlSnapShotsCount))/DATEDIFF(hour, MAX(lRow.[Timestamp]), CURRENT_TIMESTAMP)*24 [PerDay]
FROM MyGrowingTable u
LEFT JOIN
(SELECT a.[Count], b.[Timestamp], a.PerHour, a.PerDay FROM SpeedLog a
INNER JOIN (SELECT MAX([Timestamp]) [Timestamp] FROM SpeedLog) b ON a.[Timestamp] = b.[Timestamp]) lRow
ON 1=1

That is a bit much to throw at anyone all at once. To abstract it a little in pseudo code I'm saying:

SELECT CurrentCount, (y2-y1)/(x2-x1) as PerHour, PerHour*24 as PerDay
FROM MyGrowingTable
LEFT JOIN (the most recently inserted row of my log table) lRow
ON 1=1

Since my lRow sub query only returns one row ever I can get away with a 1=1 ON clause and still have one row as a result. It is important to do a LEFT join to lRow so that a first row with no rate info can be inserted.

Now I just need to insert on some regular schedule. I do this via a SQL Server Agent Job. If you've never used one of those before don't worry. It is a point and click affair within Management Studio. You just have to make sure you have SQL Server Agent running as its own service. Then you can add a new job with one task within it that runs:

INSERT INTO SpeedLog
SELECT * FROM vw_Status

Since the logic is in the view, you can tweak things without having to dive into the Agent Job every time.

Where next?

count(*) is a pretty simple measure of a table. For your own purposes maybe you want to track the average number of rows where some column is null. That would be no problem. You could track as many metrics as you wanted by just adding more and more columns to the log table and your view. More complex rate analysis could be done on the entire log table as well.

As with every solution I create. It seems like a problem other people must have run into and solved already. I think I'm just bad at finding solutions. If you have something else to tackle this problem please share!

Load HttpModule! Load! HttpModules in App_Code.

2010-02-24T11:35:00.007-07:00

I've done HttpModules in separate assemblies and don't remember integration being too tricky. But when I was just writing one within the App_Code of the same site I was having all sorts of issues. I think I found some common gotchas related to HttpModules.

First, my module was not loading but I didn't even get an error page. I was able to get useful errors by switching the App Pool of the site to "Classic .NET AppPool" from the "DefaultAppPool". Turns out this did more than give me error messages, when all was finished I had to have this classic app pool for the module to work. Thanks to lucky abhishek for figuring out the app pool problem first.

Once I was seeing errors I first found that I had other modules in a parent site that could not load. In IIS web.config settings from parent sites are inherited by sites in any virtual sub directories. So I was loading other HttpModules that I didn't care about in this sub site. I fixed this with a quick

<remove name="NameOfModuleIDontHave" />

in the HttpModules section of the web.config in order to straighten that out.

Last I just had to figure out why I "could not load type MYTYPE". Turns out for the type I was using the namespace instead of the class name qualified with the namespace. In other words in my web.config I had

<add type="ParentNamespace.SubNamespace" name="MyClassName" />

instead of

<add type="ParentNamespace.SubNamespace.MyClassName" name="MyClassName" />

As a side note. After I got this working I found for my purposes that the name attribute doesn't mean anything. I can change it to whatever and my sample HttpModule still works.

One last thing I found was that the type attribute above is usually of the form path,assembly. When you're using a class in App_Code you can use the text "App_code" as the assembly if you want to be a little more clear. So this also works:

<add type="ParentNamespace.SubNamespace.MyClassName,App_code" name="MyClassName" />

But in my case it didn't seem to be required.

For reference, this is the skeleton HttpModule I was trying to load.

The Custom Windows Toolbar - Underrated.

2010-02-19T13:22:00.005-07:00

It is one of those features that never gets any hype. Much like the quick launch bar, windows allows you to add your own custom taskbar toolbars. Just

- right click the taskbar
- Toolbars -> New Toolbar
- Point to a folder and you're all set.

This video really spells out how to create a custom toolbar. Below is a shot of mine at the moment.

There is virtually no limit to the number of shortcuts. Sub folders are supported to allow for some simple grouping within a toolbar. It makes for a much better experience than the raw directory trees I usually have to think about navigating.

Only recently has it dawned on me how to use these toolbars to really save me some time. If you're a developer you may find these ideas useful if you haven't already implemented something similar.

Visual Studio

I'm always working on different solutions in visual studio. So many so that the "Start Page" no longer is that useful (even after tricking it to show me more recently opened solutions). Plus, I hate having extra tabs open. So the first thing I do after using the start page is close it. Well if you click a shortcut to a solution file no start page opens! Ha ha, finally I've thwarted that useless tab and its "developer news" that I never read. Instead I have my "vs" toolbar.

Command Prompts

All of those visual studio solutions usually have a console application or two in them. When I want to revisit these my directory navigating gets even crazier thanks to the /bin/Debug directories that are created by default. My solution is to create batch files of the form

cmd /k cd "C:\My Solutions\My Project\bin\Debug"

I talk about this idea in another post here. The end result is my "cmd" toolbar which opens up a command prompt at whatever starting directory I want.

Remote Desktops

I am constantly hoping on and off of machines for work. My rdp toolbar beats the snot out of Start -> All programs -> Accessories -> remote desktop connection. Instead I have a saved rdp for each machine one click away.

Other Ideas

NUnit

I don't do a ton of NUnit testing but I should. A little looking around shows that nunit-gui takes command line args. You could create shortcuts to your common test fixtures.

Multiple homepages

You can pass a url to the command line of most browsers. So you can have shortcuts to multiple sites in a toolbar.

I'm sure there are even more obvious uses that I've not thought of. Please share what you've come up with.

Windows Symbolic Link Equivalent - But ghetto and not at all

2010-02-17T07:55:00.007-07:00

Say you want to create a directory that points to another directory. For example something in your home directory that point to a long and convoluted path that you often have to go down. In Linux the answer is pretty easy, the ln command. Here is a decent forum post on using the ln command.

But I'm bound to the world of Microsoft. There must be some DOS equivalent right? Well a little Googling found me a discussion on junctions. My take away is that they aren't quite the same thing, and they aren't well documented. So instead I took an approach that I feel a little bit more comfortable with.

What I ended up settling on was creating some batch files in my home directory with contents similar to

cd "C:\Some\Long\Directory\Path"

Ghetto right? But for me the combo of

- Windows Key + R

- "cmd" + Enter

- MyBatFile Enter

is all I need. With a little tab completing it is quite fast. Before I was getting pretty fast at tab completing through my entire long directory name, so this is a definite improvement from how I was operating.

You could also make shortcuts on the desktop or in a custom toolbar off the start menu of the form

cmd /k cd "C:\My\Dir"

if you like a little point and click in your life.

Finally you could get all of your shortcuts in your PATH environment variable. Then you could jump around without having be in your home directory.

I know this isn't a very impressive tip. I guess what I'm really just conveying a more philosophical lesson. Sometimes quick and dirty is the way to go. Especially when it is saving time on something as simple as directory navigating.

How to document MSSQL. Metadata vs Comment Headers

2010-02-09T10:11:00.002-07:00

I've been a fan for a long time of creating a new stored procedure the MSSQL Management Studio way.

Right click the "Stored Procedures" folder in the object explorer on the left. Select "New Stored Procedure"
Use Ctrl+Shift+M to fill in the comment header with bunches of meaningful values
Be on my merry way creating my proc

Later when I script the proc as a Create/Alter, I get my header back. Works fine for me.

But what about table and view comment headers? They don't seem to exist, although if you script a view as an alter you can inject some comments into the definition. It seems like maybe what I should be doing with my tables is outlined here, Managing Metadata. Basically you can add in metadata via the "Extended Properties" of any table. But my comment headers don't automatically appear as metadata and the metadata I enter doesn't get reflected into my comment headers. So how do I just add in descriptions to all of my database objects? I just want to a little documenting at the most basic level.

I found all sorts of elaborate ways of using metadata in enterprise applications. But it doesn't seem to be designed for the simple ways I'm trying to use it. How do you document your database objects? Am I going about this all the wrong way?

Excel Date. Excel Time. But what about DateTime?

2009-08-22T08:43:00.004-07:00

I just ran into a beef I have with Excel. But let me start by saying that I've never claimed to be an excel guru. I know a bunch of things that the average excel user doesn't. Nutty things that a non-developer likely wouldn't run into or care about. But as for analytics and number crunching, I always end up turning to MSSQL. It kills me because I know I use queries in relational databases all the time when an excel spreadsheet should be sufficient.

I have been monitoring the progress of some process in a haphazard, sanity checking sort of way. I ended up with two columns of data, the first is a timestamp, the second is an integer. I just want to see how quickly my integer is growing over time. So the most basic check would be:

(x1-x2)/(datetime1-datetime2)

But I can't find the existence of a datetime in excel. I only see a Date and then a Time. So for the denominator of the above forumula I'm now faced with

(date1-date2)*24 + time1-time2

This is assuming date1-date2 yeilds days as an integer and time1-time2 yeilds hours.

I never got any further in excel. I just wrote a query like so:

SELECT (1874797-1856322)/DATEDIFF(hour, CAST('2009-08-21 15:18:35.913' as DATETIME), CAST('2009-08-22 08:25:42.837' as DATETIME))

1086/hour ... done!

In retrospect, in excel it would have been faster for me to make a graph from my data points, add a trend line, then put a few points into that formula. From there I could get the rate I was looking for. This is me, the excel non-guru. Let me know of any good links that might enlighten me of other methods.

MSSQL uniqueidentifier: Why no aggregates?

2009-06-02T20:50:00.006-07:00

Say you have an MSSQL table with one column. It is of the type uniqueidentifier, which is a Guid.

This works:
SELECT count(*) FROM myTable

This Doesn't:
SELECT count(myCol) FROM myTable

In order to get this to work you have to do something like:
SELECT count(CAST(myCol as varchar(36))) FROM myTable

This solution and some variants I found in a number of places, while I was searching for the answer to another question. Why?

Why? Because you can't apply any aggregate function to a uniqueidentifier. The error message told me so. But why can't you do that!? Surely Microsoft could have it act like a varchar just to provide some default behavior. There must be some deeper philisophical reason.

I had to search around. I was only able to find this, indicating that the text, image, and ntext types also can't do aggregates. BUT I CANNOT FIND A REAL REASON. So someone please help.

I can understand limiting some aggregates, but count()? Makes me long for PERL, where there is always if nothing else a default behavior.

PsExec! Run things remotely

2009-06-02T20:17:00.007-07:00

PsExec saved me a ton of time (if I had found it sooner it would have saved me much much more). Let me cut to the important part before I ramble. Here is all the usage info I needed from PsExec:

PsExec [\\computer] cmd [arguments]

The complete usage is really:

psexec [\\computer[,computer2[,...] | @file][-u user [-p psswd]][-n s][-l][-s|-e][-x][-i [session]][-c [-f|-v]][-w directory][-d][-][-a n,n,... ] cmd [arguments]

So simple! I'm embarrassed to say what I tried before this. Anyways, here I go ...
My first thought was to quickly (ha!) create a web service to do what I wanted. I use this type of thinking all the time. Usually churning out a little administrative website or web service gets the job done. But I learned that creating a web service that wraps this particular exe was a HUGE pain. I tried everything but could not give IIS the proper permissions to execute it.

Then I tried impersonation to get the program to run. Has anyone ever used impersonation without hating their life? Really I'm curious. For me when impersonation seems like a solution, it means I need to stop and rethink things because impersonation is never the answer for me.

Finally, psexec came my way. I had the luxury of being on the same network as the other machines I was trying to manage. What kills me is that I had used psKill before (another tool in the appropriately names PsTools download).

The domain name is very powerful in SEO

2009-05-17T13:41:00.000-07:00

I found my domain ranking under all three of these searches today

coldblackcoffee.com - not that surprising
coldblackcoffee - also not crazy
cold black coffee - WHAT?!

I was pretty impressed with myself for ranking on what seemed like a decent keyword when I remembered I had not done any SEO optimization yet. In fact, my blog should have still been sandboxed. I'm not sure how Google found it.

So I can only conclude that the name of your domain is extremely powerful for ranking on those keywords that are similar to it. Now, the title of my blog's main page is the same as the domain name, so that is pretty huge too. But I also rank on this:

co ldbl ackcof fee

So I think Google is concatenating every search together and just looking for domain names that match. It seems to be an SEO shortcut for a small number of keywords. So don't do what I did, and buy a domain that sounds neat but really isn't relevant to your blog at all.

I Failed to Serialize the Encoding Class

2009-05-12T10:45:00.002-07:00

I got the error, "The specified type is abstract", when trying to serialize a class with a property of type Encoding (I was just tweaking existing XML). A quick look at MSDN confirms that the Encoding class is indeed abstract.

I almost accepted the fact that you couldn't serialize abstract classes until I remembered running into this issue before. You can! You just need the XmlInclude attribute.

[XmlInclude(typeof(InheritingClass1))]
[XmlInclude(typeof(InheritingClass2))]
public abstract class MyAbstractClass
... rest of class declaration

Then in the XML you can

...
<MyAbstractClassPropertyName xsi:type="InheritingClass1">
...

Unboxed Solutions has a nice article on it that shows the serialization code.

I tried using an XmlSerializer myself to create the XML and I got this error

The type System.Text.UTF8Encoding was not expected. Use the XmlInclude or SoapInclude attribute to specify types that are not known statically.

This means that in .NET they didn't bother with an XmlInclude attribute on the Encoding class for UTF8? Is it not "known statically"? I'm pretty sure it is!

So I give up. I'm going to use an enum for different encodings and then map that to the actual type that should be serializable in the first place. Anybody figure out a better way?

C# needs a cross platform GetHashCode method

2009-05-11T14:03:00.003-07:00

Here was my thought process. I need to get a hash of a string, which hashing algorithm should I use? Eh, I'll just use .NET's GetHashCode method and run with it. It should be fast and easy and that is what I'm after right now. The .NET framework is so lovely for rapid development and this will be another perfect example.

WRONG! As I soon found, the GetHashCode method can and does vary from .NET version to .NET version and also from platform to platform.

My mistake was persisting these hash codes to a database. You know ... to use as a key for some value, like a normal person might use a hash. But I had both 32 and 64 bit machines accessing the same database and they were coming up with different hash codes for the same value.

Before I go too much further, Let me say that I get the need to optimize and evolve a hashing algorithm over time. I have no issue with that. But how about giving us some overload that uses a cross platform version. For me it is far more useful to be able to persist my hashes than to have it optimized to the point where I can only use the hash codes for in memory stuff.

So for piece of mind I'm probably going to keep my own libraries for hashing. I'd like to use System.Security.Cryptography for this purpose. It seems easy enough. But can I trust this library to not have its implementation changed as well? Now I'm spooked. Maybe instead I should own the code all the way down to the bitwise ORs. Better safe than sorry.

Character set issues with Amazon S3

2009-05-11T12:49:00.002-07:00

Recently for my job I've had to start working with some wacky (to me) European characters and they've worked their way into some XML files that I'm storing in S3. Now, when I put the files into S3 things seem ok because I'm doing that via some other linux command line tool that I didn't write. But when I retrieve the file with my C# code I run into trouble.

For example, instead of the German character "ß" I get "A?". Using HTTP Analyzer I saw something similar coming back, but instead of ?s I saw random nonsense characters. Most importantly I saw 2 characters where I wanted one. ENCODING!!

Now for most HTTP request stuff I've had luck with this:

public static string GulpInputStreamAsString(HttpWebResponse response)
{
string text = string.Empty;
Encoding responseEncoding = Encoding.GetEncoding(response.CharacterSet);
using (StreamReader sr = new StreamReader(response.GetResponseStream(), responseEncoding))
{
text = sr.ReadToEnd();
}
return text;
}

But apparently S3's REST API isn't giving me a character set so instead I just had to do some guess and test and found that this works:

public static string GulpInputStreamAsString(HttpWebResponse response)
{
string text = string.Empty;
Encoding responseEncoding = Encoding.UTF8; //seems to work with German
using (StreamReader sr = new StreamReader(response.GetResponseStream(), responseEncoding))
{
text = sr.ReadToEnd();
}
return text;
}

I'm not sure who is responsible for making these encoding things less painful. You'll never be able to trust every web server, so it seems like C#'s System.Net library should have some auto detecting encoding functionality. Heck, maybe it already does. Am I missing something?