Category Archives: Develop

Improving backup to Amazon S3

I use s3.exe command line utility to backup photos to Amazon Simple Storage (S3) and it works great.

I have lots of photos to backup: 900,000 files in 4,000 folders.

Obivously it is impractical to copy all files on every backup – it will take forever. s3.exe has exact feature that I need:

/sync is used with the put command and only uploads files that do not exist on S3 or have been modified since last being uploaded, based on the timestamp. It can be used alone or in conjunction with the /sub option for a fast incremental backup of a whole directory.

Here’s command line that I use for backups:

s3.exe put mybucket C:\photos\ /sub:withdelete /sync /acl:public-read /yes /nogui

Charges

When I started using S3 however I was surpised with the bill from Amazon.

Let’s open source code to see how backup is done. Here are relevant bits:

foreach (string file in Sub.GetFiles(directory, filename, sub))
...
DateTime? lastModified = svc.getLastModified(bucket, key);
if (lastModified.HasValue && lastModified.Value > File.GetLastWriteTimeUtc(file))
{
Progress.reportProgress(key, 0, 0);
continue;
}

For each file s3.exe gets last modified date from the storage. If it is greater than last modified date of local file then no upload is performed.

Let’s do some calculations. To get last modified date s3.exe sends HEAD request. Amazon charges $0.01 per 10,000 HEAD requests. So I would end up paying $0.09 every time I perform backup. If I do it every day that’s $27 per month.

Let’s try to optimize it. How about this: for every local folder get the corresponding list of files in storage. Last modified date will be included in response. Now we’re going to issue about 4,000 LIST requests (1 for each folder). 1,000 LIST requests is $0.01 so that would be $0.04 in total. Or $1.20 per month – that’s saving of $25.80 – a big win 🙂

Making the Patch

I have created a patch by following steps from Scott Hanselman’s blog post.

I have submitted my patch to Codeplex, let’s see if project owner decides that it’s good enough to be applied.

Converting .Net DateTime to JavaScript Date

JavaScript Date constructor accepts number of milliseconds since Unix epoch (1 January 1970 00:00:00 UTC). Here’s C# extension method that converts .Net DateTime object to JavaScript date:

public static class DateTimeJavaScript
{
   private static readonly long DatetimeMinTimeTicks =
      (new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).Ticks;

   public static long ToJavaScriptMilliseconds(this DateTime dt)
   {
      return (long)((dt.ToUniversalTime().Ticks - DatetimeMinTimeTicks) / 10000);
   }
}

Usage:

var dt = new Date(<%= DateTime.Today.ToJavaScriptMilliseconds() %>);
alert(dt);

Reference

Java System.currentTimeMillis() equivalent in C#

How to enable detailed error information for IIS and ASP.NET

By default, IIS and ASP.NET hide detailed error information to prevent revealing sensitive information about your web application:
IIS generic error

ASP.NET generic error

Sometimes you need to see error details (think shared hosting). Add these entries to your web.config file to disable generic errors:

<configuration>
  <system.web>
    <customErrors mode="Off" />
  </system.web>

  <system.webServer>
    <httpErrors errorMode="Detailed" />
  </system.webServer>
</configuration>

Resources

jQuery BBQ plugin and Internet Explorer 7

Remember the problem with jQuery BBQ plugin in Internet Explorer 7? It’s not really a problem, just a minor glitch but let’s fix it anyway.

Steps to reproduce:

  1. Open BBQ demo in Internet Explorer
  2. Click Next button few times
  3. Open history menu

Observe:

No page titles, just URLs. Definitely it would be nice to see photo numbers there.

First, let’s see how Asual jQuery Address fix this problem:

_title = _d.title = value;
if (_juststart && _frame && _frame.contentWindow && 
    _frame.contentWindow.document) {
    _frame.contentWindow.document.title = value;
    _juststart = FALSE;
}

Aha! For IE7 you need to set title of the hidden iframe in addition to setting document title. So this is how you set page title:

document.title = "New title";
var iframe = $("iframe:hidden");
if (iframe.length > 0 && iframe[0].contentWindow && iframe[0].contentWindow.document)
  iframe[0].contentWindow.document.title = "New title";

Updated demo should work correctly in Internet Explorer 7.

Using jQuery BBQ plugin for photo gallery

Click here to view demo

Let’s take previous demo and use jQuery BBQ plugin instead. Our goal is Ajax photo gallery that supports browser’s back button. We will display numbers instead of actual photos for now.

$(function () 
{
  $(window).bind('hashchange', function(e) {
    var i = parseInt(e.fragment, 10);
    if (!isNaN(i))
      switchTo(i);
  });
  $(window).trigger('hashchange');
});

function switchTo(i)
{
  if ($("#number").text() == i)
    return;
  $("#number").text(i);
  document.title = i;
  if (i > 1)
    $("#lnk_prev").attr("href", "#" + (i - 1)).removeClass("disabled");
  else
    $("#lnk_prev").removeAttr("href").addClass("disabled");
  if (i < 20)
    $("#lnk_next").attr("href", "#" + (i + 1)).removeClass("disabled");
  else
    $("#lnk_next").removeAttr("href").addClass("disabled");
}

Instead of handling click event we modify href attribute of the links. BBQ will fire hashchange event whenever we "navigate" to a new page. No need to prevent default processing – simple!

One gotcha to watch out for is changing page title. If you take naive approach and simply use document.title (like I did) to set page title you won't see your titles in the history menu in Internet Explorer 7:

Compare with Internet Explorer 8:

Using Asual jQuery Address plugin for photo gallery

Click here to view demo

Let’s say you want to build photo gallery. Something like this:

You want to open next and previous photos without page reload, using Ajax. One problem with this is that browser’s back button stops working. Fortunately, it’s easy to fix with jQuery Address plugin from Asual. I tried using this plugin and it worked very well. I’ve put together a simple demo.

For simplicity, let’s display just a number instead of actual photo:

Every time you click right arrow the number increases; left arrow decrements the number by one. Simple! Back and forward buttons should work. Also, it would be nice to have different titles in the history:

The code:

$(function () 
{
  $.address.strict(false);
  $.address.externalChange(function(e)
  {
    switchTo(e.value);
  });

  $("#lnk_prev").click(function(e)
  {
    e.preventDefault();
    var i = $("#number").text() * 1 - 1;
    switchTo(i);
    $.address.value(i);
  });

  $("#lnk_next").click(function(e)
  {
    e.preventDefault();
    var i = $("#number").text() * 1 + 1;
    switchTo(i);
    $.address.value(i);
  });
});

function switchTo(i)
{
  $("#number").text(i);
  $.address.title(i);
}

The magic happens in click handler. $.address.value method changes page address and adds browser history point. You need to cancel navigation by calling e.preventDefault() — otherwise browser would follow the link and two history entries would get created.

Note that you need to disable plugin’s strict option, otherwise it would add slash symbol immediately after hash symbol, like this: #/11.

How to use ClientScriptManager.RegisterForEventValidation method

Have you ever encountered this error:

Invalid postback or callback argument.  Event validation is enabled using <pages enableEventValidation=”true”/> in configuration or <%@ Page EnableEventValidation=”true” %> in a page.  For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them.  If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.

That’s pretty long error message. I get this exception if I add items to drop-down control from JavaScript. The question is how to use ClientScriptManager.RegisterForEventValidation method?

First, let’s reproduce the problem. Create a new website and copy/paste this code:

<asp:DropDownList ID="dd" runat="server">
  <asp:ListItem>One</asp:ListItem>
  <asp:ListItem>Two</asp:ListItem>
</asp:DropDownList>
<asp:Button ID="Button1" runat="server" Text="Test" />

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"
  type="text/javascript"></script>
<script type="text/javascript">
  $(function()
  {
    $("#<%= dd.ClientID %>").append($("<option />")
      .val(3)
      .text("Three!"));
  });
</script>

If you select option “Three” sure enough, exception is thrown. To prevent it you need to supply all possible values for drop-down control:

protected override void Render(HtmlTextWriter writer)
{
  Page.ClientScript.RegisterForEventValidation(dd.UniqueID, "3");
  Page.ClientScript.RegisterForEventValidation(dd.UniqueID, "4");
  Page.ClientScript.RegisterForEventValidation(dd.UniqueID, "11");
  // and so on
  base.Render(writer);
}

The exception is fixed but server variable for drop-down control is useless – it has no idea that you’ve added new item so dd.SelectedValue will give One, not 3. You need to read POST variable directly instead, like this:

protected void Page_Load(object sender, EventArgs e)
{
   if (IsPostBack)
      Response.Write(Request.Form[dd.UniqueID]);
}

How to view whitespaces in Emacs

Here’s how you can view whitespaces in Emacs:

M-x whitespace-mode RET

By default, Emacs uses bright colours to highlight whitespaces that are in wrong place (in Emacs’ opinion):

To disable colouring add the following to your .emacs file:

; disable colours in whitespace-mode
(setq whitespace-style '(space-mark tab-mark))

Now it’s much better:

I assign showing white spaces to Ctrl+Shift+8 to mimick Visual Studio behaviour:

(global-set-key (kbd "C-*") 'whitespace-mode)

How to insert new GUID with one key press

When I work with WiX source code I need to create lots of GUIDs. In my Visual Studio 2005 editor all I need to do is to press Alt+G and new GUID magically appears at the cursor. Achieving this is very easy:

  1. Open Macros IDE by selecting Tools | Macros | Macros IDE.
  2. Create new macro:
  3. Public Sub InsertGuid()

    DTE.ActiveDocument.Selection.Text = Guid.NewGuid().ToString()

    End Sub

  4. Bind new macro to Alt+G keyboard shortcut. Open Options dialog, select Environment, then Keyboard on the left. Type “InsertGuid” to locate the macro you just created. Assign Alt+G to it.

    Fortunately, no standard command is assigned to Alt+G by default.

How to automate Adobe InDesign CS3

1. Locate Resources for Visual Basic.tlb in C:\Documents and Settings\All Users\Application Data\Adobe\InDesign\Version 5.0\Scripting Support\5.0 folder.

2. Open command prompt and execute:

TlbImp.exe "Resources for Visual Basic.tlb" /out:Interop.InDesign.dll

TlbImp.exe is located in C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin folder on my machine. This will generate Interop.InDesign.dll file. This is an interop assembly.

3. Add reference to Interop.InDesign.dll to your project.

4. Use this code:

using System;
using InDesign;

namespace HelloInDesign
{
   class Program
   {
      static void Main(string[] args)
      {
         object missing = Type.Missing;
         Type type = Type.GetTypeFromProgID("InDesign.Application.CS3");
         if (type == null)
         {
            throw new Exception("Adobe InDesign CS3 is not installed");
         }
         _Application app = (_Application)Activator.CreateInstance(type);
         Document document = (Document)app.Documents.Add(true, missing);
         Page page = (Page)document.Pages[1];
         TextFrame textFrame = page.TextFrames.Add(missing,
            idLocationOptions.idUnknown, missing);
         textFrame.GeometricBounds = new string[] { "6p", "6p", "24p", "24p" };
         textFrame.Contents = "Hello from .Net!";
      }
   }
}