Update on WHS2SmugMug

Back in January i said I’d get back to writing this Windows Home Server Add-In.

Its now June, 6 months later.  For 3 of those months my camera was back at Nikon being repaired. So I took exactly zero pictures during that time. Its now back and I’m bracing myself for the flood of pictures. I carry 26Gb’s worth of memory cards with me, so I nearly always end up over doing it.

Which brings me to the Add-In. I’ve set up a Codeplex page here. And I’ve made a few check-ins. This is not even pre-alpha code. Let me explain.

A few weeks ago, I asked, via Twitter,  Omar Shahine if I could use his Smugmug Uploader code. now I’m a great fan of the Uploader. I’ve used it for every single uploaded to SmugMug.

So Omar kindly emailed me his code.

So what you will find in the Check-ins, should you be brave enough to take look is Omar’s back-end code sans any UI as part of a WCF service. The WCF service is hosted by a Windows Service (imaginatively called “UploadService”). My plan ( cunning or not) is to have the UI in the Console communicate with the uploader process via WCF. There are other methods, but WCF gives me incredible latitude when it comes to moving data back and forth.

So the Home Server Console tab will simply be a UI for uploading stuff. Instead of Remoting in and using Omar’s uploader. This is an intermediate goal.

My ultimate goal is actually to have a “Smugmug” folder and under it a folder for every Smugmug Album in your account. the above mentioned service will monitor those folders for changes and replicate those changes to Smugmug.

So I’m building now with such a convoluted architecture with a view to where i want to get this Add-in to.

So hopefully I can get the Add-in working soon.

I’m a pretty good test case for this, but I will need testers for it.

Watch my twitter account or my FriendFeed account for updates 😉

PS If you’re asking why I’ve not moved blogs yet, I’m waiting for the next Oxite Release first.

Designing a new Blog Header

So I’m designing a new blog, as per my previous post.

I’m NOT a graphic designer. But I am a photographer.  And I get the fact that the design of the blog has got be linked to the content.  My point being that a landscape or a nature scène looks out of place when you’re discussing the finer points of programming languages or social networks.

On the other hand, you can’t always predict what you’re going to blog about ( at least in my case), so you want to be general in some way.

If you’re following me on Twitter or Friendfeed, you’ll see that I’ve been posting alot of the stuff I’ve found on web design in general.

So, brimming with inspiration, I’ve gone off and trawled through my photo archives for something relevant.

So here are a few that I’m thinking of using in a big way,  as the header, footer, or both (i.e. cutting the picture in half):

Engineering:

 

Nasa:

Bright Spark:

All the images you see here are on Smugmug.

Let me know what you think.

Back and ready to rock

So I’m back from holiday (Florida – no ride queues and was great).

And I’m raring to go .

To start the year off, WHS4Smugmug development will resume ( after a year of being busy and feeling guilty). There’s a great series of posts on Add-In development on the tentacleBlog:

And I’ll be using them to bootstrap development and hopefully get moving.

One area I’m worries about is the file/folder structure that it’ll pull the photos from and send them to Smugmug.

So I’ll crowdsource this problem. Please leave a comment on how you organise your photo folder. Thanks.

This’ll help me with the 8000 photos I took in Florida.

FriendFeed Gets My Images, Too – FriendFeed Notify 0.2

Like this picture?

(Yerba Buena Island – Thomas Hawk)

Me too. How about this one:

(Passage- johopo)

Nice huh?

One more:

(Untitled – Me)

Couldn’t resist.

My point is that the above three images will be posted to FriendFeed along with the link to this post by the new release of FriendFeed Notify 0.2.

Now this isn’t for just for photography buffs like me and Thomas Hawk. It works for any images embedded in an img tag and greater than 50x100px.

Now the release isn’t actually feature complete. There are a few things I’d like to add to it. These will be in the 0.2.1 release. Including picking and choosing which images to post and  the posting of a comment by way of summary. These are simple to implement and I don’t thing it will be too long before they are out.

So go and get it from here.

Anyone looking at the code will see that i am using the .Net frameworks webbrowser control to retrieve images. This runs in the same dialog you are shown the images.The regexes I tried are all in the code, but commented out. If anyone can help with these, that would be great. It would cut down on the overhead. Thanks.

For those of you reading this and wondering hat happened to my Smugmug add-in for WHS.its been on the back burner for a while. I didn’t expect the hiatus to take this long. In the meanwhile, Omar Shahine has updated his Send To SmugMug utility. Some of the features for the next release that people are voting on are similar in concept to my add-in. So, go vote. I still intend to do this Add-In and get it integrated with WHS.

Enjoy.

WHS Add-In: WHS2SmugMug – Update

I’ve neglected this project for a while mainly due to me being so busy with other stuff.

The hiatus has actually done the project good as feature creep was threatening to de-rail the thing the last time I had a look at it.

So I’ve cleaned up the requirements for the data to be stored locally. I’ve eliminated just about everything I can pull from SmugMug leaving me with a nice, clean object model to work with.

I was also struck by the fragmenting of the project into three – scheduled, service and WHS Console Add-in.

While this seems logical, it is a bit over the top. So I’m dropping the scheduled uploader and having uploads handled by the service.

Work is progressing nicely and I hope to have a working  service app soon ( if not a console add-in).

I’m now using the SmugMugAPIWrapper from Codeplex. Its MIT Licenced so WHS2SugMug will have to be too. This library is one that I can actually use without looking at the source as its built to use the current SmugMug API, so no worries there.

As with my Windows Live Writer add-in, I’ll host the project on Codeplex as soon as there is a release-ready codebase.

FriendFeed

After all the hype, I’m on FriendFeed.

Finally. In addition to my blog posts showing up there, I’m sharing my Link Blog and my SmugMug Photostream.

Here’s my page and the feed is in an unassuming text label under Feeds in  the column to your right. I’m going to try find or make a proper label for it, since it looks out of place at the moment.

It will be interesting to see which feed people prefer, the blogs’ feed or the FreindFeed one.

Its Been a While – WHS Add in

I’ve been off ill. Pretty badly in fact, which is why  haven’t posted. or done much programming.

I’m adding the finishing touches to the service – mainly making sure it won’t crash itself or the Server and that it actually does what I intend it to do.

So I’m gradually turning my attention to the scheduled task that will upload files. This is a nightmare mainly because of the large number of possible scenarios. These mainly have to do with updated files. It depends what has changed – the image itself (i.e. after a trip through Photoshop or Elements) or the metadata (keywords, tags, etc).

I’m also getting to grips with the settings and console tabs in terms of what will go where. I tend to try to nail down the user interface first in my software projects so that I know what user interface logic needs to be implemented. No use writing lines of code that, although doing something quite logical and correct, turn out not to be needed.

I’ve yet to forget about this little project- just wish Omar Shashine would release his code for his Send to Smugmug App. Don’t forget to check his site for the latest version.

WHS and SmugMug- Architecting the Add-In

The structure of an Add-In is important to what we are trying to accomplish here.

I could put all my code as part of a Console Tab and have people manually getting it working.

That would no doubt work, but its kind of lame and falls short of expectations.

So we want an application that reacts in real-time to folder and file changes, but is also able to be scheduled ( say for 3am after a defrag or back-up session).

This requires a number of different application blocks, each an application in its own right.

To see how an Add-In works in code, the WHS Photo Uploader for Flickr is hosted on Codeplex. Unfortunately its a totally different application for a totally different service. But I did get a good look at the Console and Settings Tabs to see what the code would/should look like.

If you want to download and run it, I’d download the WIX Installer XML for Visual Studio 2005 to allow the  Add-in’s installer to be recognised by VS and compiled.

So the lesson is that Console Code is run from the Console (doh!) and any file monitoring code will only work while the console is running.

So we’ll require a service to be running to monitor our gallery folders and record the events to our Master XML file. Hence this application will consists of XML code and our FileWatcher Object. It will need to use the Windows API to allow it to be installed and run  as a service.

Then, we want to schedule our uploads so that they are not taking over your broadband connection. We could use System.Threading to calculate the number of seconds remaining and have the timer call our upload method once the time is reached.

However, we want to keep things as simple as possible. This means using windows’ Scheduled Tasks to call our uploader (an exe file). Scheduling Tasks is for some reason not in the .Net Framework 2.0 so we’ll use a library to wrap all the C++ Interop code. There is a CodeProject article here on it with the library. This also means we can run tasks with a users username and password (i.e “Administrator” and whatever the password is) so tasks will run, in theory, even when no one is logged in.

So we have three applications here, all rather trivial:

  • The Console Tab and Settings Tab with code for usernames/passwords as well as rescheduling uploads and manually starting the uploader. I have the idea of getting album stats and displaying the charts by using the Google Charts API, but that’s for version 1.1.
  • The Service app that will take file and folder change events and writes them to our Master Settings file. This runs independent of any user being logged in.
  • The Uploader that is scheduled to run at such and such a time to execute the events in our Settings file on the server using he SmugMug API. Also runs independent of any users being logged in.

Slowly but surely the Add-in progresses.

WHS and SmugMug – A Word About XML

XML is a markup language in the same family as HTML. This means you have nested elements in a  structured document.

Its structure makes it ideal to read and write data quite easily and in  a logical format. Since the elements all have names, the documents are usually human readable. This is of course a disadvantage for a proprietary file format.

Reading and writing XML is quite easy with Visual Studio and the .Net Framework. You can use an object called XMLSerializer that will do it automatically, but it depends on you structuring you data for it. It is also great for recurring data. In the example below we are reading in multiple FileEvent object.

The way I prefer doing it is using XMlReader and XMLWriter. Like so:

reader.Read();

 for(int loops =0; loops<count;loops++){
                    FileEvent temp = new FileEvent();
                    reader.ReadStartElement();
                    temp.FilePath=reader.ReadElementString();
                    temp.EventType = reader.ReadElementString();
                    reader.ReadEndElement();
                    mastersettings.Events.Enqueue(temp);

}

reader.Close(); 

Line for line, the read and write methods of your application is exactly the same as your final XML document, unless you use a loop.

Once you know how to use XmlReader and Writer, its easy to use. Essentially you have to ensure that your elements open and close at the right points it the code to ensure that you get a properly formatted Xml Document.

Second, make sure you are reading the right data type. I use strings as they are quite easy to use, even for dates and such. Most of my errors come from, that.

Third, make sure that you read and write methods read and write the exact same data at the exact same points ( you don’t want to be reading a name when you expect a date, etc).

Fourth, you need to use the WriteStartDocument method to start writing an Xml file and WriteEndDocument for ending one. thing of them as a huge node that encapsulates rest of the document. Just remember, one document per file.

And that’s all there is to it.

More later on how I’m using XML as a data source.

Related Post: WHS and SmugMug – Keeping Track of Files

WHS and SmugMug – Keeping Track of Files

The question that has to be asked at the beginning of every software project, hobbyist or commercial is: “What are we trying to achieve here?”.

For this project, the aim is to move all the contents of one folder to a gallery on SmugMug – and keep them up-to-date. This means that modifications ( tags, edits, etc), additions and deletions are replicated on the gallery in question.

This requires two things:

  • The program to be aware of what is happening to those folders
  • And for the program to be aware of what has already been uploaded.

Number one requires a FileWatcher Object from System.IO.

The Filewatcher throws an event for creations, deletions  and modifications. This allows us to determine the nature of, say, a create event and decide if its a file or directory that has been created. Based on that we can take other action.

We do this by calling System.IO.Directory.Exists(string path) that returns a boolean. You could use the File.Exists(string path) method as well.

Second, a running tally has to be kept of what is going on. We do that by maintaining a master file in our root directory (say \\Photos\SmugMug)  and child files in each directory. The actual directory may or may not be exposed as a setting via the Console.

The master file is essentially has a queue of file events to execute on the server the next time its syncs. This stops large file  ( i.e. large numbers of files) operations overloading the server (resulting in multiple create events) as Demigrator.exe is working on those files as well. So better  to queue the uploads to take place when the disks are idle. I’m using XML and handwriting the read and write procedures – you’ll understand why at the end of the post.

The second requires a file in each directory with the SmugMug Gallery ID and  a list of each file and its SmugMug ID in that Directory. This allows us quick access to files instead of iterating over the lot and comparing each file to the one on SmugMug.

As far as scheduling the uploader is concerned, I haven’t gone into that in too much detail. I could schedule an exe to run using the task scheduler. Its not a bad way of doing things. Norton Internet Security does just that for its scheduling. That, of course has a disadvantage of spitting up the add-in into a logical front-end and back-end.

And by the way, I’m writing this in Visual Studio 2005 using C#. I’m going to do a second version in C# Express 2008 (to use LINQ-to-XML, mainly) for a comparison.