Setting up Visual Studio Online and Hipchat Integration

While searching for a way to add TFS integration to Hipchat, I came across a bunch of posts that were not particularly useful. In my particular case, I was looking to integrate Visual Studio Online's TFS with Hipchat, so using a separate service to manage this integration seemed like unnecessary pain. So, I poked around a little in the Visual Studio Online portal and found that it is indeed quite easy to set this up. Here are the steps I followed, more as a reminder to myself for the next time I want to do this, but hopefully it will help others as well.

First, log in to your hipchat account and go to the home page. If you have already created a room that you want to integrate with, then this is where you will want to be and you can skip to the next paragraph. If you haven't created the room yet, you'll have to do that before you do anything else. And guess what, you can't do that from the group admin home page, so go ahead and launch the web app. Once in the chat app, create the room and go back to the home page.

Now that you have the room with which you want to integrate, from the home page, click on the rooms tab and then click on the room you want to integrate with. Now we need to create a token, so click on the tokens link and create a new token. You'll need this token for the next step.

That is all the setup that is required on the Hipchat side. Let's now log in to Visual Studio Online. Once you are at the home page, select the project you want to integrate with. We want to update the settings for the project, so click on the gear icon on the top right. Now, click on the Service Hooks tab and create a new service hook. Select Hipchat from the service list that comes up and then select the type of trigger you want. At some point you'll get asked for the token you created earlier. Copy that into the Authorization Token field and then finish up whatever else you need to in there. At the end you'll have the option to send a test message to confirm that everything is working.

That's it. No service to install or plugin to configure. I have to say it was much easier than I expected.

Introducing the Quartz.Net Feature Pack

I'd like to introduce the Quartz.Net feature pack project to you. It's hosted on GitHub and it's main purpose is to provide some features that are not currently part of Quartz.Net. Traditionally, Quartz.Net has been a direct port of the Java implementation of Quartz and so it doesn't typically add new functionality that might be useful if it is not part of the Java version.

Believe it or not, work on this project goes on, albeit slowly, because of the usual reasons. I did want to use this post to outline what is already there and also to keep a running list of things I'd like to see. These features are all tied to some sort of pain point that I've run into as I've been using the scheduler. Let's start with what is already there, in some shape or form.

What is Currently Available in the Quartz.Net FeaturePack

What is Not There Right Now But I'd Like to See

  • All of the above items finished (some are almost done, some are quite basic)
  • A way to dynamically reload job DLLs, so the scheduler doesn't have to be stopped and started whenever you update one of the jobs.
  • Cluster aware management tools
  • Nicer graphical interfaces. Yes, there are a few out there but they're scattered all over. Would be nice to have everything in one place.

I'm making a conscious effort this year to spend more time working on this project. If you have any suggestions on features that you would like to see added, send me an email or just head on over to the QuartzNetFeaturePack project page on Github and add an issue there.

Quartz.Net Built-In Listeners

This is the fifth post in the Quartz.Net Listener Tutorial series. In this post we'll be looking at the listeners that are shipped with the Quartz.Net distribution.

Scheduler Listeners

If we search through the Quartz.Net codebase we'll find two classes that implement the ISchedulerListener interface: SchedulerListenerSupport and BroadcastSchedulerListener. Let's look at these guys in detail now.

SchedulerListenerSupport

The documentation for the SchedulerListenerSupport says that this class is

A helpful abstract base class for implementors of ISchedulerListener

The methods in this class are empty so you only need to override the subset for the ISchedulerListener events you care about.

Well, there you have it. An empty class where all the methods of the interface are implemented but left empty. I guess this can save you some typing but I'm not sure it's very useful if you want to implement a listener. The way to use this is to inherit from this class and then only implement the methods you really are interested in.

BroadcastSchedulerListener

The documentation for the BroadcastSchedulerListener says that this class

Holds a List of references to SchedulerListener instances and broadcasts all events to them (in order).

This may be more convenient than registering all of the listeners directly with the Scheduler, and provides the flexibility of easily changing which listeners get notified.

If we were to hide all of the methods that are required by the ISchedulerListener interface, then we would be left with these two methods, which are at the heart of what this listener does:

public void AddListener(ISchedulerListener listener)
{
    listeners.Add(listener);
}

public bool RemoveListener(ISchedulerListener listener)
{
    return listeners.Remove(listener);
}

The listeners variable is a generic List, so it follows the semantics of that class. In essence this listener manages a list of listeners and then forwards all of the scheduler events to all the listeners. This is probably useful if you have a large number of scheduler listeners.

Now let's go hunting for some job listeners. I think we might find some interesting things there.

Job Listeners

After searching through the codebase for classes that implement IJobListener and classes that inherit from JobListenerSupport, I was able to find 4 job listeners: JobListenerSupport, BroadcastJobListener, JobChainingJobListener and LoggingJobHistoryPlugin. Let's look at these job listeners now.

JobListenerSupport

The documentation for JobListenerSupport states that this class is

A helpful abstract base class for implementors of IJobListener.

The methods in this class are empty so you only need to override the subset for the IJobListener events you care about.

You are required to implement IJobListener.Name to return the unique name of your IJobListener.

As you can see, it's a class similar to the SchedulerListenerSupport class we reviewed earlier. It's an empty implementation, so perhaps useful if you're only interested in some of the methods and want to save some keystrokes. It also includes a logger instance in it. The JobChainingJobListener that we're about to discuss inherits from this instead of implementing the IJobListener interface directly.

BroadcastJobListener

The documentation for BroadcastJobListener states that this class

Holds a List of references to JobListener instances and broadcasts all events to them (in order).

The broadcasting behavior of this listener to delegate listeners may be more convenient than registering all of the listeners directly with the Scheduler, and provides the flexibility of easily changing which listeners get notified.

This class is also similar to the BroadcastSchedulerListener we went over. Similarly, it provides just a few methods in addition to the ones required by the interface:

public void AddListener(IJobListener listener)
{
    listeners.Add(listener);
}

public bool RemoveListener(IJobListener listener)
{
    return listeners.Remove(listener);
}

public bool RemoveListener(string listenerName)
{
    IJobListener listener = listeners.Find(x => x.Name == listenerName);
    if (listener != null)
    {
        listeners.Remove(listener);
        return true;
    }
    return false;
}

Like its scheduler counterpart, this broadcast listener forwards all of the job events to all the registered job listeners. If you have lots of job listeners, this could be useful.

JobChainingJobListener

The documentation for JobChainingJobListener states that this class

Keeps a collection of mappings of which Job to trigger after the completion of a given job. If this listener is notified of a job completing that has a mapping, then it will then attempt to trigger the follow-up job. This achieves "job chaining", or a "poor man's workflow".

Generally an instance of this listener would be registered as a global job listener, rather than being registered directly to a given job.

If for some reason there is a failure creating the trigger for the follow-up job (which would generally only be caused by a rare serious failure in the system, or the non-existence of the follow-up job), an error messsage is logged, but no other action is taken. If you need more rigorous handling of the error, consider scheduling the triggering of the flow-up job within your job itself.

Kind of cool no? A simple way to chain jobs is provided in the box for you. It's not a fully developed workflow engine but for sequential jobs it probably does the trick. Let's look at the two methods of interest in this job listener. Here they are:

public void AddJobChainLink(JobKey firstJob, JobKey secondJob)
{
    if (firstJob == null || secondJob == null)
    {
        throw new ArgumentException("Key cannot be null!");
    }
    if (firstJob.Name == null || secondJob.Name == null)
    {
        throw new ArgumentException("Key cannot have a null name!");
    }

    chainLinks.Add(firstJob, secondJob);
}

public override void JobWasExecuted(IJobExecutionContext context, JobExecutionException jobException)
{
    JobKey sj;
    chainLinks.TryGetValue(context.JobDetail.Key, out sj);

    if (sj == null)
    {
        return;
    }

    Log.Info(string.Format(CultureInfo.InvariantCulture, "Job '{0}' will now chain to Job '{1}'", context.JobDetail.Key, sj));
    try
    {
        context.Scheduler.TriggerJob(sj);
    }
    catch (SchedulerException se)
    {
        Log.Error(string.Format(CultureInfo.InvariantCulture, "Error encountered during chaining to Job '{0}'", sj), se);
    }
}

The first method, AddJobChainLink, is the method we would call to set up the links. That's all we need to do to set up a chain: tell the scheduler which job should fire (secondJob) after the first job completes (firstJob). The JobWasExecuted method is called by the scheduler once a job has finished running. If the listener finds a link for the job that was executed, it will trigger the new job immediately.

This is one of those features that not many people know about but it comes up every now and then on the mailing list.

LoggingJobHistoryPlugin

The documentation for LoggingJobHistoryPlugin states that this class

Logs a history of all job executions (and execution vetos) via common logging.

The logged message is customizable by setting one of the following message properties to a string that conforms to the syntax of string.Format(string,object).

JobToBeFiredMessage - available message data are:

Element Data Type Description
0 String The Job's Name.
1 String The Job's Group.
2 Date The current time.
3 String The Trigger's name.
4 String The Triggers's group.
5 Date The scheduled fire time.
6 Date The next scheduled fire time.
7 Integer The re-fire count from the JobExecutionContext.

The default message text is "Job {1}.{0} fired (by trigger {4}.{3}) at: {2:HH:mm:ss MM/dd/yyyy}"

JobSuccessMessage - available message data are:

Element Data Type Description
0 String The Job's Name.
1 String The Job's Group.
2 Date The current time.
3 String The Trigger's name.
4 String The Triggers's group.
5 Date The scheduled fire time.
6 Date The next scheduled fire time.
7 Integer The re-fire count from the JobExecutionContext.
8 Object The string value (toString() having been called) of the result (if any) that the Job set on the JobExecutionContext, with on it. "NULL" if no result was set.

The default message text is "Job {1}.{0} execution complete at {2:HH:mm:ss MM/dd/yyyy} and reports: {8}"

JobFailedMessage - available message data are:

Element Data Type Description
0 String The Job's Name.
1 String The Job's Group.
2 Date The current time.
3 String The Trigger's name.
4 String The Triggers's group.
5 Date The scheduled fire time.
6 Date The next scheduled fire time.
7 Integer The re-fire count from the JobExecutionContext.
8 String The message from the thrown JobExecution Exception.

The default message text is "Job {1}.{0} execution failed at {2:HH:mm:ss MM/dd/yyyy} and reports: {8}"

JobWasVetoedMessage - available message data are:

Element Data Type Description
0 String The Job's Name.
1 String The Job's Group.
2 Date The current time.
3 String The Trigger's name.
4 String The Triggers's group.
5 Date The scheduled fire time.
6 Date The next scheduled fire time.
7 Integer The re-fire count from the JobExecutionContext.

The default message text is "Job {1}.{0} was vetoed. It was to be fired (by trigger {4}.{3}) at: {2:HH:mm:ss MM/dd/yyyy}"

The documentation is pretty extensive but the bottom line is that it's a listener that logs all of the job related events to the common.logging infrastructure. It's pretty useful if you're not sure why jobs aren't firing when you expect them to and to get a general idea of what the scheduler is doing.

The one thing that might strike you as odd is that it's also a scheduler plugin. This is so that you can add the job listener through the configuration file upon scheduler startup so that you don't miss any events. There are a couple of ways of adding listeners to a scheduler, but I'll adress that in a different post.

Trigger Listeners

The last set of built-in listeners that we'll take a look at are the trigger listeners. Let's search the codebase for ITriggerListener and see what we find. I only found 2 classes that implement the ITriggerListener interface: TriggerListenerSupport and BroadcastTriggerListener. Just like we did for job and scheduler listeners, let's go a little deeper and talk about these classes a bit.

TriggerListenerSupport

The documentation for the TriggerListenerSupport says that this class is

A helpful abstract base class for implementors of ITriggerListener.

The methods in this class are empty so you only need to override the subset for the ITriggerListener events you care about.

You are required to implement ITriggerListener.Name to return the unique name of your ITriggerListener.

This is the third time we've seen a description such as this one, and I think that by now you get the idea. This is, for the most part, an empty class. There's a logger there and the VetoJobExecution will always return false.

BroadcastTriggerListener

The documentation for the BroadcastTriggerListener says that this class

Holds a List of references to TriggerListener instances and broadcasts all events to them (in order).

The broadcasting behavior of this listener to delegate listeners may be more convenient than registering all of the listeners directly with the Scheduler, and provides the flexibility of easily changing which listeners get notified.

You've seen this before, yes? It's another broadcasting implementation that forwards trigger events to trigger listeners that are registered with it.

What was that all about?

We've just gone over all of the listeners that come with the Quartz.Net distribution. Now you know that there are empty implementations of all 3 listeners that you can inherit from if you only want to implement a few of the methods required by the interface. You also know that there are 3 broadcasting listeners that you can use if you have large numbers of listeners that you want to manage a bit easier. Finally, we found a couple so interesting job listener classes that could come in handy if you wanted to get more detailed information on what was going on inside the scheduler (LoggingJobHistoryPlugin) or if you want to link together multiple jobs, so that they execute in sequence (JobChainingJobListener).

I hope this is useful information that may come in handy some day. If you wish to comment on this post, send me an email and I'll attach your comment to this post.

Quartz.Net Trigger Listeners, Part 4 of Quartz.Net Listeners in Detail

This is the fourth post in the Quartz.Net Listener Tutorial series. It’s also the fourth part of the introduction to listeners overview series. You can find Part 1 here, Part 2 here and Part 3 here. Today we’ll be looking at trigger listeners and how to implement one.

As we mentioned in Part 1, trigger listeners get notified of trigger level events. To implement a trigger listener, you need to implement the ITriggerListener interface. Here’s what that interface looks like:

1 public interface ITriggerListener
2 {
3     string Name { get; }
4     void TriggerFired(ITrigger trigger, IJobExecutionContext context);
5     bool VetoJobExecution(ITrigger trigger, IJobExecutionContext context);
6     void TriggerMisfired(ITrigger trigger);
7     void TriggerComplete(ITrigger trigger, IJobExecutionContext context, SchedulerInstruction triggerInstructionCode);
8 }

You’ll notice that trigger listeners get notified when the trigger is fired (line 4), when the trigger misfires (line 6) and when the trigger is complete (line 7). There is one other method that also gets called on the trigger listener that needs special attention. When the VetoJobExecution (line 5) method is called, the trigger listener can tell the scheduler to ignore the trigger. Essentially it vetoes the firing of the trigger.

Now let's build a trigger listener that implements all of these methods. Download the sample code from the Quartz.Net ebook if you want to follow along. Here's the code for the trigger listener we'll be going over:

 1 using Quartz;
 2 using System;
 3 using System.Reflection;
 4 
 5 namespace Examples
 6 {
 7     class TriggerListenerExample:ITriggerListener
 8     {
 9         public void TriggerFired(ITrigger trigger, IJobExecutionContext context)
10         {
11             Console.WriteLine("The scheduler called {0} for trigger {1}", MethodBase.GetCurrentMethod().Name, trigger.Key);
12         }
13 
14         public bool VetoJobExecution(ITrigger trigger, IJobExecutionContext context)
15         {
16             Console.WriteLine("The scheduler called {0} for trigger {1}", MethodBase.GetCurrentMethod().Name,trigger.Key);
17             return false;
18         }
19 
20         public void TriggerComplete(ITrigger trigger, IJobExecutionContext context, SchedulerInstruction triggerInstructionCode)
21         {
22             Console.WriteLine("The scheduler called {0} for trigger {1}", MethodBase.GetCurrentMethod().Name, trigger.Key);
23         }
24 
25         public void TriggerMisfired(ITrigger trigger)
26         {
27             Console.WriteLine("The scheduler called {0} for trigger {1}", MethodBase.GetCurrentMethod().Name, trigger.Key);
28         }
29 
30         public string Name
31         {
32             get { return "TriggerListenerExample"; }
33         }
34     }
35 }

This is a very simple listener, as it only logs any calls made to it, but it's quite useful because it provides insight into which methods are called when.

First, take a look at the TriggerFired method (line 9), since it is the very first method that gets called on the trigger listener. After the TriggerFired method gets called, the VetoJobExecution (line 14) gets called. You'll notice that our implementation returns true always. If this method returns false, then the JobExecutionVetoed method will get called on the job listener (if there are any) and the job will not get executed.

Next, let's look at the TriggerComplete method (line 20). This method gets called when trigger has completed, which means that it gets called after the trigger has fired and after the associated job has finished executing.

The TriggerMisfired method (line 25) will get called if the trigger ever misfires, so it's a little out of place, because it's not really related to the job processing lifecycle.

Finally, let's look at the Name property (line 30). Not much excitement here, as it just returns the name of the listener.

Now that we've reviewed the code for the listener, let's take a look at the listener in action. Here's the output generated by running the scheduler with the configured examples.

 1 The scheduler called SchedulerStarting
 2 The scheduler called JobAdded for job directoryScanJobExample.directoryScanJobExample
 3 The scheduler called JobScheduled for job directoryScanJobExample.directoryScanJobExample, caused by trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
 4 The scheduler called JobAdded for job directoryScanJobExample.addDirectoryScanListener
 5 The scheduler called JobScheduled for job directoryScanJobExample.addDirectoryScanListener, caused by trigger directoryScanJobExampleSimpleTriggerGroup.addDirectoryScanListenerSimpleTrigger
 6 The scheduler called JobUnscheduled
 7 The scheduler called JobScheduled for job directoryScanJobExample.directoryScanJobExample, caused by trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
 8 The scheduler called JobUnscheduled
 9 The scheduler called JobScheduled for job directoryScanJobExample.addDirectoryScanListener, caused by trigger directoryScanJobExampleSimpleTriggerGroup.addDirectoryScanListenerSimpleTrigger
10 The scheduler called SchedulerStarted
11 IsStarted=True
12 SchedulerInstanceId=NON_CLUSTERED
13 SchedulerName=ServerScheduler
14 The scheduler is running. Press any key to stop
15 The scheduler called TriggerFired for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
16 The scheduler called TriggerFired for trigger directoryScanJobExampleSimpleTriggerGroup.addDirectoryScanListenerSimpleTrigger
17 The scheduler called VetoJobExecution for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
18 The scheduler called VetoJobExecution for trigger directoryScanJobExampleSimpleTriggerGroup.addDirectoryScanListenerSimpleTrigger
19 The scheduler called JobToBeExecuted for job directoryScanJobExample.directoryScanJobExample
20 The scheduler called JobToBeExecuted for job directoryScanJobExample.addDirectoryScanListener
21 Job addDirectoryScanListener in group directoryScanJobExample is about to be executed
22 Job directoryScanJobExample in group directoryScanJobExample is about to be executed
23 The scheduler called JobWasExecuted for job directoryScanJobExample.addDirectoryScanListener
24 The scheduler called JobWasExecuted for job directoryScanJobExample.directoryScanJobExample
25 Job addDirectoryScanListener in group directoryScanJobExample was executed in 29ms
26 Job directoryScanJobExample in group directoryScanJobExample was executed in 33ms
27 The scheduler called TriggerComplete for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
28 The scheduler called TriggerComplete for trigger directoryScanJobExampleSimpleTriggerGroup.addDirectoryScanListenerSimpleTrigger
29 The scheduler called TriggerFinalized for trigger directoryScanJobExampleSimpleTriggerGroup.addDirectoryScanListenerSimpleTrigger, belonging to job directoryScanJobExample.addDirectoryScanListener
30 The scheduler called JobDeleted for job directoryScanJobExample.addDirectoryScanListener
31 The scheduler called TriggerFired for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
32 The scheduler called VetoJobExecution for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
33 The scheduler called JobToBeExecuted for job directoryScanJobExample.directoryScanJobExample
34 Job directoryScanJobExample in group directoryScanJobExample is about to be executed
35 The scheduler called JobWasExecuted for job directoryScanJobExample.directoryScanJobExample
36 Job directoryScanJobExample in group directoryScanJobExample was executed in 5ms
37 The scheduler called TriggerComplete for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
38 The scheduler called TriggerFired for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
39 The scheduler called VetoJobExecution for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
40 The scheduler called JobToBeExecuted for job directoryScanJobExample.directoryScanJobExample
41 Job directoryScanJobExample in group directoryScanJobExample is about to be executed
42 The scheduler called JobWasExecuted for job directoryScanJobExample.directoryScanJobExample
43 Job directoryScanJobExample in group directoryScanJobExample was executed in 5ms
44 The scheduler called TriggerComplete for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger
45 The scheduler called TriggerFinalized for trigger directoryScanJobExampleSimpleTriggerGroup.directoryScanJobExampleSimpleTrigger, belonging to job directoryScanJobExample.directoryScanJobExample
46 The scheduler called JobDeleted for job directoryScanJobExample.directoryScanJobExample
47  Shutting down scheduler
48 The scheduler called SchedulerInStandbyMode
49 The scheduler called SchedulerShuttingdown
50 The scheduler called SchedulerShutdown
51 IsShutdown=True
52 The scheduler has been shutdown.

This listing includes all of the output from the listeners we've covered so far, so it gives us an example of how it all comes together. The first few lines (1-14) are output by the scheduler listener as the scheduler gets started and the jobs get scheduled. Once the scheduler has started, we can see that there are 2 TriggerFired method calls (lines 15 and 16). One runs the directory scan job and the other one basically "installs" the directory scan listener. Next, the scheduler calls the VetoJobExecution method on both trigger listeners (lines 17 and 18). You'll notice that we only created one listener but it's getting called for both triggers. That's because our listener was added with a matcher that matches all of the triggers. Take a look at the PluginExample file if you're not sure about what's going on.

Since our trigger listener returns false when the VetoJobExecution method is called, the scheduler then executes the jobs attached to the triggers. After the jobs have finished running, the TriggerComplete method is called for both triggers (lines 27 and 28). This means that the triggers are done for now. One of the triggers is not supposed to run again, so the scheduler calls the TriggerFinalized method on the scheduler listener (line 29). This trigger won't fire again. Because the job that is attached to that trigger is not set to persist, then the scheduler will delete the job from the schedule as well (line 30).

We still have one trigger that needs to keep firing, so you'll notice that the cycle will repeat for this trigger (lines 31 through 44). The TriggerFired method is called, then the VetoJobExecution, then the TriggerComplete method is called. Finally, the trigger's repeat count is zero and then the scheduler will call the TriggerFinalized method on the scheduler listener (not the trigger listener!) and delete the job (lines 45 and 46).

I hope seeing this all in one place makes it come together for you. We'll stop here for now and in our next post we'll go over the Quart.Net built-in listeners.

Moving from Wordpress to Jekyll

If you visit my blog frequently, you may have noticed that the layout of the site has changed drastically. Can you guess why? Yes, I switched from using wordpress to using Jekyll.

Why switch?

Speed

Basically because I wanted my site to load faster. I moved off shared hosting and into Azure to speed up the site and there definitely was an improvement. However, it's pretty hard to beat a static site, so I'm going with static for now.

More content

As Jekyll has a blogger importer, I was able to import my old posts and now they're all available on this site.

Less maintenace

No more backing up MySql and updgrading Wordpress

What I'm Giving Up

Comments

For now I'm giving up on comments but I may add them later. All in all I don't get many comments.

Forms

I'm also giving up on forms right now but may also add them later. I don't need more spam anyway.

Final Thoughts

It took a while to get everything moved over and getting the feel for how things work. I also discovered I had some broken links left over from the previous migration to Azure. Those should all be fixed now, but do let me know if I missed any.

Now I'll get back to writing some more posts.