Coding

Depot, an exercise in Community-Sourcing

Gerry Heidenreich — Sun, 08 Jun 2008 04:43:59 GMT

No downloads or pics, just a quick rundown of a very cool app idea while it's in my head.

About a year and a half ago, I wrote a small winforms app. It's stayed very rough around the edges and hasn't gone anywhere from the original prototype. This prototype (I called it Depot) was written as a proof-of-concept of the simplest possible community-sourced bookmarking / tagging / searching tool that could possibly exist. A self-organizing business-specific link / text library could provide immense value to a company.

Depot hinges on 4 basic features common with collaborative apps:

1. Producing: Adding content in the form of URLS and/or text (2 different fields that can be used individually or combined)

2. Tagging

3. Searching for any item by any combination of title words or tag

4. Sharing: All content is automatically shared, and open to edit & extend, by anyone within the network

The search is an autocomplete textbox, that works with any combination of title words and tags. Typing 'catering' displays all catering items, but as you start to type 'catering madison', the suggestions filter appropriately. As you would expect, changing the text over to 'thai madison' updates to items tagged or titled with thai and madison.

The url + text fields is an interesting feature - a user may want to toss in a quick note for a catering url someone else added, like "Beware the red curry!!!".

The app seemed to work beautifully, but the algorithm is not built to scale up yet. Everything is cached heavily on the client-side. There are no concurrency checks. Also, to be fit for production, it will need some kind of user-auditing, history, and probably some kind of browser integration (or at least bookmark / favorites sync).

I don't know yet what will become of Depot. I hope to find the time and motivation soon to dust it off and start polishing it up for a pilot group. If nothing else, I got an ornery hog of a tag-search algorithm that may come in useful someday.

Stackoverflow.com = joelonsoftware + codinghorror

Gerry Heidenreich — Thu, 17 Apr 2008 03:37:58 GMT

Jeff Atwood has paired up with Joel Spolsky and today they both announced stackoverflow.com, a new community support site that doesn't exist yet... but will be run by Jeff, created by the community for the community, and narrated weekly via podcast. Looks like a digg-like structure where diggable items are answers to questions posted by the community. Submit questions, submit answers, vote for answers...

[Jeff's announcement]

[Joel's announcement]

[stackoverflow.com]

[podcast #1]

This will be interesting...

The Thirsty Developer, Project Euler

Gerry Heidenreich — Fri, 11 Apr 2008 16:28:31 GMT

I can't believe that I didn't post about this! My site was acting up when we did this, so I probably didn't have a place to post it.

Larry Clarkin & Dave Bost from Microsoft have a podcast called The Thirsty Developer.

Back in December I met up with Larry, Dave, and Damon Payne at the Ale House in downtown Milwaukee to talk about Project Euler. I remember having a lot of fun, despite being a bit nervous (my first experience on that side of a podcast!). It was very loud in there as well; I think Larry did a great job of editing.

[Here is the Project Euler episode]

[The Thirsty Developer]

[Subscribe to The Thirsty Developer Podcast]

[Subscribe to The Thirsty Developer Blog]

EnterpriseLog, usage-logging

Gerry Heidenreich — Tue, 01 Apr 2008 23:03:50 GMT

Most development work contains some type of logging. Usually, it's to a local log file or an email, for exception tracking at the most.

Recently, I decided to throw together an EnterpriseLog table. We tossed our logging bits into a central library... I don't know why it took so long to centralize this particular functionality, but I'm glad we did it.

Our EnterpriseLog table is providing useful data, as well as helping us answer some good questions:
- What applications are people using?
- Tracking ClickOnce downloads, system updates made by clickonce apps, etc.
- What terms are people searching for?
- Exceptions
- Usage

Usage Logging
Usage logging is has been very interesting so far. An immediate benefit we have found is keeping track of how formatted information is being entered in unexpected ways. It is amazing how many different ways there are to enter a phone number! With a quick look at EnterpriseLog usage data for an application, we are able to recognize, learn, and accommodate actual usage patterns. Continuing with the phone # topic:

With any phone number, a quick regex levels the playing field by stripping out all non-digit characters:
string nums = Regex.Replace(textBox.Text, @"[^\d]", "");

If the result string is 10 chars (in the US), format appropriately for area code and exchange. If it is 7 characters, try to infer the area code from the exchange. If it is only 4 characters, assume it is an extension and build the real number from there if possible.

In this case, usage logging helped me to skip the annoying validation and instructions and let my users do what they do. We have been able to identify actual usage, throw assumptions out the window, and help them arrive at the appropriate result invisibly:
textBox.Text = String.Format("({0}) {1}-{2}", areaCode, exch, num);

Most of us base our UI development on assumptions about our users, past experience, and hopefully a book or two. It is the assumptions part I have trouble with. I'm a data guy. I prefer to question the assumptions, set up some auditing and collect some data, and turn the assumptions into concrete UI features that keep the user in the flow.

Salesperanto AND Coderian?

Gerry Heidenreich — Fri, 14 Mar 2008 16:03:47 GMT

"Stop Thinking Like A Programmer"

I swear it's the theme this week.. I was actually told this by someone at work. In my own defense, I was thinking like somebody that would rather script a solution than wait 3 weeks for it (ok yeah, that's like a programmer)....

Think about this...

1. Business: Coders should be able to think in terms of features, interface complexity, barrier to entry, design and visualization, and capable of elevator pitching their product (notice I didn't say solution?) to a customer in these terms.

2. Innovation: getting the "I can do that" people (e.g. your engineers/architects/coders) to be able to speak directly to the "it would be cool if..." people (e.g. your billers, customers, parents, etc)

3. Apple's innovation/momentum and Microsoft's shift in perspective & ability to compete: Microsoft has always been guilty of "thinking like programmers", and it has been very profitable for them, but things are changing, and they are reacting accordingly.

4. Black & White: on one side are the geeks that appreciate your architecture and could debate code/frameworks/paradigms all day. On the other side are your customers, who want to know how you are going to make them more profitable/efficient/confident/marketable/competitive. Not much of a grey area here. 2 different languages: Salesperanto AND Coderian.

5. Intentional Programming [wikipedia]: Your skillset is in demand, but we are getting closer to the day that "Everyone Writes Software"... Lutz has a section dedicated to this. Developers must learn to understand the intent of their users. Stop thinking in syntax, start thinking in semantics... Mashups, FDD, REST, RDF, Pipes & Popfly (Google SETS prediction).

There is a pattern developing here, and there is A LOT of money being tossed around because of it (check this out!).

.Net now "Shared Source" NOT Open Source

Gerry Heidenreich — Wed, 03 Oct 2007 16:49:00 GMT

Scott Guthrie announced on his blog, a few hours ago, that .Net source code will be opened up to the public, under a ms-rl license (for reference, read-only).

Scott's announcement, already flooded with lots of comments & trackbacks, mostly positive, is here.

Starting with Visual Studio 2008 (Orcas), currently set to be released later this year, we will be able to reference the internal state of .Net objects as if they were local.

This means a few things:
1. F11 will step you into the actual .Net object being called, where you can reference in-state .Net classes.
2. You will see real objects, variables, line numbers in your call stack for the .Net classes being referenced.
3. (I assume) you will be able to “Go to definition“ and view actual .Net class source code instead of an interface. Of course, we've always had the ability to reflect on the libraries, and with a little work, figure out what was happening... but this will make things a lot more simple and accessible (have a look at Reflector to figure out the Asp.Net treeview control!)
4. WWBAD (What would Brad Abrams do?) Now we can see for real instead of reverse engineer it and spend our time figuring out what the variable datetime17 is doing. In other words, quality of code should improve. As we constantly reference the .Net library, some of the msft QA, v3.5 juju should rub off on us and help us fall a little more in line with standards, best practice, etc.

Anyway, it's read-only. It's not Open source. But it's another big step in what I think is the right direction for Microsoft...

Some good MS pages to check out on open/shared source initiatives:
- Open Source at Microsoft
- Microsoft Shared Source Initiative
- MS Developer Tools (A goldmine of open/shared source projects, most on CodePlex)

Profile your ASP.Net 2 apps with nProf

Gerry Heidenreich — Fri, 14 Sep 2007 15:15:00 GMT

I'm going to make this short and sweet - there's not much [non-Polish (?!) ] documentation out there for nProf, so here are my notes for getting it to hook into your ASP.Net 2 app and show you some bottleneckage:

Preparation:

1. Set processModel username:
Open your machine.config file (C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config), hit Ctrl+I (incremental search) and type processModel, hit F3 (~2 times) until you find the <processModel...> tag. Add the userName attribute within it: userName="SYSTEM"

2. Make sure your app is hosted in and accessible via IIS (nProf won't hook into Cassini) - ex: http://localhost/test

Install nProf:

There are a bunch of versions available, and they're all alphas... The one that worked for me is here (nprof-0.9.1-setup.exe on sourceforge).
Install the .exe, and when prompted go ahead and run nProf.

Profiling:

Note: DO NOT stop profiling via nProf. Instead, recycle your app pool, or restart IIS inside your IIS manager. nProf will see it's recycled and stop for you. If you use nProf's stop buttons, it will hang and you'll lose your profile data.

File > New, Create Profiler Project dialog opens, select ASP.NET radio button, click Create Project.

Click Project > Start Project Run (or just hit F5)

Open browser, browse to IIS-hosted asp.net app (use IIS address, ex, http://localhost/nProfTest). Browse through pages that are slow, or save separate profiles for individual pages. You will see nProf's Messages window start rolling with information.

Once again, don't click 'Stop Run' in nProf! Instead, go to IIS Manager, and either recycle your app pool, or restart IIS. This will properly stop nProf, and you will see your resulting profile data.

Analysis:

Back in nProf, you will see the threads called by your aspnet_wp.exe process, and when you select them, you will see the used namespace hierarchy below. Find the thread with your library profile. Uncheck the parent namespaces and re-check your own. On the right, you will see the method signatures called, number of times called, % time taken, and more. Click the '% of Total' header to sort, and you will see your bottleneck methods at the top.

Note: Saving & Opening is kludgy: You can try to save, but I've had no luck re-opening my profiles... instead I took a screenshot of my trouble methods just for reference.

Intelligent Image Resizing - Amazing video demo!

Gerry Heidenreich — Thu, 23 Aug 2007 13:10:00 GMT

Via Digg > OhGizmo!, the video & article is here: Smart Image Resizing Cuts The Useless Out Of Your Pics

I'll check out the 20mb (pdf) whitepaper sooner or later, but here's what's going on as far as I can tell...

Technique:
This engine is resizing images by removing paths of pixels vertically or horizontally instead of columns of pixels. The path they add/remove is determined by interestingness. They refer to this technique as “retargeting“ (as opposed to resizing or resampling).

Identifying Image Interestingness:
They are determining the interestingness (or in their terms: gradient magnitude) by starting with a grayscale version of the source image, then creating a new image using The Sobel Operator on the source image. The resulting image represents large gradient activity (interestingness.) with light, and boringness... with dark.

Original grayscale, and the Sobel gradient image of the same picture:

Discovering the least-interesting path:
The engine then uses an 'importance or energy function' to discover the least-interesting path, composed of 1px from each column or row. This function seems to represent the most consistently shaded 1-pixel path from one end of the image to the other...

“Retargeting” the image:
Once the least-interesting path is discovered, it is then either removed or copied, depending on if you are “retargeting” your image to make it larger or smaller.

Maybe ImageManipulator needs some Retarget methods... now if only optionsscalper could help me work through this math ;)

Here's the video:

Euler12 (spoiler warning) - GetDivisors

Gerry Heidenreich — Thu, 03 May 2007 18:19:00 GMT

Thanks to Mr. Payne, I've been spending more hours awake coding, and less time sleeping, in the name of 'fun and recreation'...

As Damon points out in his esoteric and colorful post yesterday, I've been working my way through the Euler (prounounced 'oy ler!) puzzles at Project Euler. I've banged out solutions for puzzles 1-13, but #10 is the only one I can't get under 1 minute (In fact, it's the only one that isn't solved almost instantly).

Problem 12 is interesting: Which is the first triangle number to have over 500 divisors?

Finding triangle numbers is trivial; the actual challenge in this puzzle is efficiently counting divisors for increasingly huge numbers.

The initial approach that almost works:
For number n, iterate from 1 to n, increment divisorCount where i % n == 0
(Once you try this out, you will find that with this approach, your highest divisor count will quickly reach 320 divisors, and will ultimately hang at around 480 before it starts to crawl...)

Finding the first triangle number to have over 1000 divisors, in under 10 seconds:
When things get even slightly complex, I turn to pen & paper... within minutes I saw the pattern I was looking for. The following method will return a list of divisors for an int:

static List<int> GetDivisors(int num) {
   List<int> divisors = new List<int>();
   int divisorCandidate = 1;
   int multiple = 1;
   do {
      if (num % divisorCandidate == 0) {
         multiple = num / divisorCandidate;
         divisors.Add(divisorCandidate);
         //for squares, divisorCandidate == multiple (3*3=9 etc), so only count it once
         if (divisorCandidate < multiple) divisors.Add(multiple); 
      }
      divisorCandidate++;
   } while (divisorCandidate < multiple);
   divisors.Sort();
   return divisors;
}

I doubt it's even remotely useful in production, but this method is a great example of how a little analysis of the problem (especially away from the keyboard) can result in a much more elegant solution to your problem than brute force.

Pasting Code into TiddlyWiki

Gerry Heidenreich — Sat, 14 Apr 2007 17:29:00 GMT

Precede code with {{{ and a newline... end code with a newline & }}}