Automate EBS Snapshot in Amazon EC2 with CSharp

23. November 2011 22:22

Ok, so I've had this code for a long time now (over a year) and have never had a chance (or forgot) to post this here.  This is basically a solution I came up with to help automate my EBS snapshots in my Amazon EC2 environment.  The code does the following:

  • Create snapshots for volume names specified in the code
  • Delete snapshots that are 2 days old for the volume names specified in the code

I have ripped out code from my overall solution. I have in my code, additional functionality for logging and more volumes, but this sample should get you started and on your way.

You will need the following for this to work:

This code is created as a console application. You can leave it like that or switch it to a windows service, so you don't have to worry about it as much. Also, this code could be optimized so feel free to do so, and post your comment here if you'd like.

The zipped filed contains all the code you will need. I'll give a short explanation of each file

EC2\BaseEC2.cs -> This is the base class that contains the property that creates your EC2 client based off of your keys

EC2\Snapshot.cs -> This contains the following methods

public void MaintainSnapshots()

        /// <summary>
        /// Returns information regarding the snapshot ID
        /// </summary>
        /// <param name="snapshotId"></param>
        /// <returns></returns>
        private DescribeSnapshotsResult DescribeSnapshot(string snapshotId)

        /// <summary>
        /// Return the snapshot ID based off the snapshot description filter
        /// </summary>
        /// <param name="snapDescription"></param>
        /// <returns></returns>
        private string GetSnapshotIDByDesc(string snapDescription)
        
        /// <summary>
        /// Returns a collection of snapshot results based off a list of snapshot ID's
        /// </summary>
        /// <param name="lstSnapshotIDs"></param>
        /// <returns></returns>
        private DescribeSnapshotsResult DescribeSnapshot(List<string> lstSnapshotIDs)
        
        /// <summary>
        /// Creates a single snapshot
        /// </summary>
        /// <param name="name"></param>
        /// <param name="volumeID"></param>
        /// <returns></returns>
        private CreateSnapshotResult CreateSnapshot(string description, string volumeID)
        
        /// <summary>
        /// Deletes a single snapshot
        /// </summary>
        /// <param name="snapshotId"></param>
        /// <returns></returns>
        private bool DeleteSnapshot(string snapshotId)
 
 

EC2\Volume.cs -> contains and enum for the volume names and 2 methods

 

/// <summary>
/// Returns a single volume ID givien the volume name provided
/// </summary>
/// <param name="volumeName"></param>
/// <returns></returns>
public static string GetVolumeID(XDevVolume volumeName)

/// <summary>
/// Returns a list of volumes Id given the specified volume names in the list provided
/// </summary>
/// <param name="volumeNames"></param>
/// <returns></returns>
public static List<string> GetVolumeID(List<string> volumeNames)
 

App.config -> where you enter in your keys

Program.cs -> Main method that just calls the MaintainSnapshots method from the object Snapshot

And that's it. If you guys have a different way of doing so please let me know. I'm sure this is not the best way, but gets the job done for a simple task like this.

As always use this code at your own risk, test and modify as needed to fit your solution

Download Solution Below

 

XDev.AWS.rar (9.77 kb)

Tags:

Amazon Cloud

jQuery Modal Dialog Helper in ASP.NET Web Pages

10. November 2011 19:26

Now that I have been working on a few new projects (which I'm very exited to get launched), I've decided to write them using ASP.NET Web Pages. I am using Visual Studio instead of WebMatrix.  I steered away from Web Forms this time around to try something new and see how it goes. So far so good. I am loving ASP.NET Web Pages and find it very fast and powerful and easy to integrate javascript with data I would normally have trouble with in Web Forms. I will admit, I do miss the whole "code-behind" concept from Web Forms. However, either MVC or architecting out the ASP.NET Web Pages app can make up for it pretty easily.

Anyway, Helpers in Web Pages I have found to be very effective. If any of you use jQuery or jQuery UI, then you understand how awesome they both are.  I created simple Helpers to show a jQuery UI Modal Dialog and I hope you find them useful. They are easy to implement on your web page and easy to implement as a Helper. 

This assumes you have a reference to the propery jQuery scripts either on a layout page or the page you will be using these helpers.  I have made two of them. One for a success message and one for an error message.  You'll notice image reference. They are basically a chosen success icon and error icon, so replace as needed.

Code for the helpers

@helper GetSuccessMessage(string dialogID, string autoOpen, string title, string message)
    {
    <script type="text/javascript">
        $(function () {

            $("#@dialogID").dialog({
                autoOpen: @autoOpen,
                height: 200,
                width: 500,
                modal: true,
                title: '@title',
            });
        });
    </script>
    
    <div id="@dialogID" style="padding: 10px 10px 10px 10px">
        <table cellpadding="5" cellspacing="5" style="width: 100%; border: 1px solid green;">
            <tr>
                <td valign="top">
                    <img src="@Href("~/images/success.png")" alt="success" />    
                </td>
                <td valign="top">
                    @message
                </td>
            </tr>
        </table>
    </div>
    
}

@helper GetErrorMessage(string dialogID, string autoOpen, string title, string message)
{
    <script type="text/javascript">
        $(function () {

            $("#@dialogID").dialog({
                autoOpen: @autoOpen,
                height: 200,
                width: 500,
                modal: true,
                title: '@title',
            });
        });
    </script>
    
    <div id="@dialogID" style="padding: 10px 10px 10px 10px">
        <table cellpadding="5" cellspacing="5" style="width: 100%; border: 1px solid red;">
            <tr>
                <td valign="top">
                    <img src="@Href("~/images/error.png")" alt="error" />    
                </td>
                <td valign="top">
                    @message
                    <br /><br />
                    If problem the persists please email <a href="mailto:something@something.com">support@something.com</a> with a description of your issue.
                </td>
            </tr>
        </table>
    </div>    
}

Now this is how you would use them in your Web Page.

@SiteHelpers.GetSuccessMessage("dialog-success", @isValid.ToString().ToLower(), "", @confirmationMessage)

@SiteHelpers.GetErrorMessage("dialog-error", @isError.ToString().ToLower(), "", @confirmationMessage)

 

//TODO: rest of code above to perform any action
if(yourVariable)
{
    isValid = true;
    isError = false;
    confirmationMessage = "Item has been updated.";

}                    
else
{
    isValid = false;
    isError = true;
    confirmationMessage = "Error updating item. Please try again";
}

Well I hope you find these jQuery UI Modal Dialogs easy to use in your ASP.NET Web Pages application.  Now that's I've done two full apps, I found myself using them all the time.

References

http://jquery.com/

http://jqueryui.com/demos/dialog/

http://www.asp.net/webmatrix/tutorials/3-creating-a-consistent-look 

 

Tags:

ASP.NET Web Pages | Javascript

Setting Folder Permissions for Umbraco

30. September 2011 16:36

This is going to be a quick post. I am starting to mess around with Umbraco and seeing that it could be very useful in certain situations.  Anyway, this post is literally showing you the batch file I used to set permissions on folders.  I gathered all of this from a few posts, as each one was slightly different in my eyes. So I figured, I might as well add another one out there for those that may be having problems.

I installed 5 different Umbraco sites (version 4.7.1) locally and they all ran under the same Application Pool in IIS. This was done on Window 7 Ultimate (64-bit). 

The structure looks like this:

C:\inetpub\wwwroot\umbracodev
C:\inetpub\wwwroot\umbracoblog

Etc….

First thing I did was create the batch file below:

icacls app_code /grant "IIS APPPOOL\umbracodev":(OI)(CI)RX
icacls app_browsers /grant "IIS APPPOOL\umbracodev":(OI)(CI)RX
icacls app_data /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls bin /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls config /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls css /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls data /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls masterpages /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls macroscripts /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls media /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls python /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls scripts /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls umbraco /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls usercontrols /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls xslt /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls web.config /grant "IIS APPPOOL\umbracodev":(OI)(CI)M
icacls web.config /grant "IIS APPPOOL\umbracodev":M

Next I copied this file into each root of each site (c:\inetpub\wwwroot\umbracodev\UmbracoPermission.bat)

Then I went into each site and right clicked the bat file -> Run as Administrator

Then you should see something like this from your output

 

This worked for me, and I hope it works for you.  Now I can get started on working with this CMS!

Here are some links I referenced when doing this:

http://our.umbraco.org/wiki/install-and-setup/set-umbraco-folder-permissions-from-command-line
http://our.umbraco.org/wiki/install-and-setup/how-to-install-umbraco-on-windows-server-2008
http://blog.mattbrailsford.com/2010/08/01/adding-a-windows-context-menu-item-to-set-umbraco-folder-permissions/

Tags:

Umbraco

Using Plupload in ASP.NET

7. July 2011 18:21

Test Solution Download

PLUploadTest.rar (121.28 kb)

I like plupload, however, using it for a real world application was a bit tough my first time around.  I basically wanted to have the user upload a file, then record that record in the database.

So I made a skeleton version of the code that you can use to help get you started. This will allow you to use Plupload in your asp.net application to do the following:

  • have the user select a file (or files) via plupload
  • Have the user submit a file(s)
  • Have a generic handler be aware of the id's of a given object(s) (i.e., a user id)
  • Store the file uploaded into a given directory
  • The ability to store the id's of an object(s), file id and file name from plupload

To accomplish this you will need the following:

  • Plupload
  • Custom javascript (not too much) you will see a file named xdev.plupload.js in js/plupload
  • Enable Page Methods via the ASP.NET ScripManager

 

Once you have a skeleton version of the code you like, I think you'll find it easy to modify and incorporate plupload into your project as well. I love this multifile upload, so why not use it effectively in your website application :)

I will bascially do a copy and paste of the ASP.NET code so you can see what's going on.  I think the best thing about this is the ability to take the files and the ID of your object (i.e., user id) that is uploading them and storing the data in a database.

PLUploadTest.aspx

In the "head" part of the aspx page you will need to reference the following javascript and css

    <script type="text/javascript" src="https://www.google.com/jsapi">
    <script type="text/javascript">
        google.load("jquery", "1.3");
    </script>
    <script src="js/plupload/plupload.full.min.js" type="text/javascript">
    <script src="js/plupload/jquery.plupload.queue.min.js" type="text/javascript">
    <script src="js/plupload/xdev.plupload.js" type="text/javascript">
    <link href="plupload.queue.css" rel="stylesheet" type="text/css" />

In the body section of the ASPX page you will need the following:

    <form id="form1" enctype="application/x-www-form-urlencoded" method="get">

        <div id="uploader" style="width: 525px;">
            <p>You browser doesn't have Flash, Silverlight, Gears, BrowserPlus or HTML5 support.

</div> <br /> <div style="width: 150px; margin-right: 490px;"> <a onclick="SubmitItem(this)" href="#">Submit Item </div> </form>

PLUploadTest.aspx.cs

[WebMethod]
public static string GetIDs()
{
    //This is strictly here just so you can see how you can grab ID's and make plupload aware of them
    //This is useful for data driven application. Example, you need the ID of the user who is uploading a file
    //so you can pull that data out later when the user logs in.

    //default is pipe delimited for js file in function OnGetIDSucceeded
    //IDOfObject1|IDOfObject2
    string IDs = "0|0";

    //These IDs could be pulled from anywhere (i.e, Session, DB, XML file)

    IDs = string.Format("{0}|{1}", "10", "11"); //"10" and "11" and filler so you can have data to mess around with

    return IDs;
}


[WebMethod]
public static void InsertFileRecord(string idOfObject1, string idOfObject2, string fileID, string fileName)
{
    _InsertAttachment(idOfObject1, idOfObject2, fileID, fileName);
}



private static void _InsertAttachment(string idOfObject1, string idOfObject2, string fileID, string fileName)
{
    //TODO: Insert into Database as needed, or whatever data store you are using.
    string doSomething = ""; //left here for debugging purposes...delete as needed
}

UploadHandler.ashx.cs

 

public void ProcessRequest(HttpContext context)
{
    //TODO: Whatever you would like to do now that you have the ID's of the objects
    //Example, could be creating directories for that ID specifically, etc....        
    string _idOfObject1 = context.Request.Headers["IDofObject1"].ToString();
    string _idOfObject2 = context.Request.Headers["IDofObject2"].ToString();

    int chunk = context.Request["chunk"] != null ? int.Parse(context.Request["chunk"]) : 0;
    string fileName = context.Request["name"] != null ? context.Request["name"] : string.Empty;

    //TODO: change as needed for your application
    string uploadPath = context.Server.MapPath("~/documents/"); 

    HttpPostedFile fileUpload = context.Request.Files[0];

    fileName = fileName.Insert(0, string.Format("{0:MM}{0:dd}{0:yyyy}", DateTime.Now));

    using (var fs = new FileStream(Path.Combine(uploadPath, fileName), chunk == 0 ? FileMode.Create : FileMode.Append))
    {
        var buffer = new byte[fileUpload.InputStream.Length];
        fileUpload.InputStream.Read(buffer, 0, buffer.Length);

        fs.Write(buffer, 0, buffer.Length);
    }

}

public bool IsReusable
{
    get { return false; }
}

 

 

One of the custom javascript methods that handles the files and data. Please download the zip file to see the entire solution.

 

function SubmitItem(e) {

    //get a copy of the current uploader
    var uploader = $('#uploader').pluploadQueue();
    var obj = uploader.settings;

    //alert("Local js variable ID1: " + IDofObject);
    //alert("Local js variable ID2: " + IDOfObject2);

    //alert("Header Object1: " + obj.headers.IDofObject1);
    //alert("Header Object2: " + obj.headers.IDofObject2);

    // Validate number of uploaded files
    if (uploader.total.uploaded == 0) {
        // Files in queue upload them first
        if (uploader.files.length > 0) {
            // When all files are uploaded submit form
            uploader.bind('UploadProgress', function () {
                if (uploader.total.uploaded == uploader.files.length)
                    $('form1').submit();
            });

            //now go insert the files into the DB
            uploader.bind('UploadComplete', function (up, f) {

                var obj = up.settings;

                //alert("Upload Complete");

                for (i = 0; i < f.length; i++) {

                    //alert("Name: " + f[i].name);
                    //alert("ID: " + f[i].id);

                    InsertFileRecord(obj.headers.IDofObject1, obj.headers.IDofObject2, f[i].id, f[i].name);
                }

                alert("File(s) Uploaded"); //You may not want this. I just left it here so you can see how it interacts with everything else.

            });

            uploader.start();
        }
        else {

            alert("No files to upload"); //take out or do whatever you would like here because there were zero files
        }
    }

}

 

Resources

http://www.plupload.com/
http://www.plupload.com/example_queuewidget.php
http://www.plupload.com/example_custom.php
http://www.plupload.com/example_events.php
http://www.plupload.com/plupload/docs/api/index.html

 

 

PLUploadTest.rar (121.28 kb)

Tags:

ASP.NET | Javascript

How to Install the EC2 API Command Line Tool for Windows

17. February 2011 01:38

Getting the EC2 Command Line API Tools to work on windows was beneficial for me so I thought I would share it with others. I found it to be beneficial because I was in a situation where I wanted to change the instance size one one of my Windows instances. You can't accomplish this from the amazon management console, but you can from their api command line tool.

This was installed on Windows 7 Ultimate 64-bit


First step is to download the tools from the following URL

http://aws.amazon.com/developertools/351?_encoding=UTF8&jiveRedirect=1


Once you have the tool downloaded, extract the file to a directory of your choosing.  I chose to extract the tool into C:\EC2\

Second step is that you will need your x.509 certs from your amazon account.

To accomplish this

  • I went to my "account" page within AWS Management Console
  • Clicked on Security Credentials
  • Under access credentials sections I clicked Create new certificate and downloaded the private key and certificate when prompted too.


Note: Keep track of where you downloaded them. I just put them in my C:\EC2\ directory to keep things in one place


Third, you will need to make sure you have the Java SDK installed.  I downloaded it here


http://www.oracle.com/technetwork/java/javase/downloads/index.html


Fourth, you will need to enter Environment Variables on your Windows system.

I entered in the following environment variables

Variable: EC2_CERT
Value: Path to your cert file name you got from amazon
Example: C:\EC2\cert-XYXYXYXXYXYXY.pem


Variable: EC2_HOME
Value:  C:\EC2\

This needs to be the location where the api tools are

Variable: EC2_PRIVATE_KEY
Value: Path to the file of your private key you got from amazon
Example: C:\EC2\pk-XYXYXYXYXYXXYXY.pem

Variable: JAVA_HOME
Value: C:\Program Files\Java\jdk1.6.0_23\

This needs to be the path where you installed the Java SDK


This should do it. You should now be able to run the following command to make sure it is working properly:

ec2-describe-regions

 

To change the instance type of an instance, first stop the instance, then run the following command

ec2-modify-instance-attribute yourInstanceID -t m1.xlarge

In the command above replace yourInstanceID with your actual instance id and then change m1.xlarge to the actual size you want it to be.

Then start your instance back up and you are good to go with the new size.


Here is a link to the API reference doc.

http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/

Tags:

Amazon Cloud




My Random Thought

I think the OCW is a great thing to have available to those who are in school, just finished school or just want to educate themself

http://ocwconsortium.org/

John On Twitter

Discounts