Flattr this Bookmark and Share

Thursday, March 17, 2016

A Different Definition of Agile Software Projects

"Agile" must surely be the most used (and abused?) buzzword ever in Software Development circles!  But what does it really mean? 

I don't think there is a single correct definition, but I can explain what it means to me and why just to give you another perspective.  I have formed this opinion after reading and listening to many other people's opinions (such as Jergen Appelo) but reworded it so that it makes sense to me and helps me realize some actual benefits from the concept.  Please feel free to leave comments if you agree or disagree because the reason for writing this up is to see if it resonates with other people or if you think I am way off track.
To me, if a Software Project is considered to be "agile" it means that all the stakeholders are happy and if they become unhappy, they are made happy again in a timely fashion.
nootn.com.au - 2016

What?  Happiness?  Let me explain from a couple of different perspectives.  If you don't feel like reading the whole text maybe just skim the "TL:DR" sections in italics to get the gist of what I am talking about.

Customers/Clients/Product Owners/Project Managers *

*I will refer to all the types of people listed as just "client" for simplicity - it may be any or all of the above in your scenario. 

TL:DR - Trying to be "Agile" may encourage us to rush into things and not think about longer term goals. Ensure that the client is going to be "happy" in the long run not just getting them short term happiness that may turn into unhappiness later.

I think where some definitions of "Agile" fall short is they elude to the fact you need to please the customer, but try to be too specific about the "software" aspect.  For example definitions along the lines of "Agile means being able to implement changes and new features quickly without breaking things" don't really explain why that is important, or if it really is all the time.  It is very specific to the quality of the code base - ways to achieve that are having high code coverage and robust software architecture following the SOLID principles for example.  This is great, but it doesn't necessarily address any aspects of the Software Development Process or that particular project being "Agile", just the code base.

A common request from a client might be "I want this new feature in our app quick because our competitor is talking about introducing it soon".  If you are blindly trying to be "Agile" you might throw that on the top of your backlog and work on it in the next sprint.  This might make the customer temporarily "happy" because they feel like they will not be lagging behind, but they might lose sight of the bigger picture or other lost opportunities.  If you have an agile process, it might involve taking the time to do it better than the competitor and making sure something else more important that will get longer term goals does not drop off the list, or blow out the budget.  If your process can ensure that the best long term outcome may be achieved by not rushing into that feature request, and you can explain that to the customer, then they may end up still being happy in the short term, but also in the longer term.  You may be "Awesomely Agile" according to my definition by doing nothing more than explaining a feature is a bad idea and not implementing it - if that makes the client happy and longer term keeps them happy then you are Agile!  Yay!

Product Team (Developers, Testers, BA's etc)

TL:DR - In order to achieve the objective above of keeping the Customers/Clients/Product Owners/Project Managers happy, the team should be happy and remain happy also.  If they are unhappy, they are unlikely to be able to keep other stakeholders happy and may drag the project down to a point where it is longer able to be "Agile".
Most projects start out new and exiting and people are generally happy to begin with - so it's only natural that things can go south from there since the bar is raised fairly high to begin with!  Initially the team is probably burning through tasks and delivering a lot of benefit in a short time.  Everyone on the project team is happy.  Then things start to go off the rails.  Maybe aspects of the process are not working well, clients are changing their mind and wasting effort or not communicating effectively.  A very common one is that code starts to "smell" as we rush things and make changes so we quickly end up with a lot of technical debt.  If nothing is done about this then even if the team can keep the client happy in the short term, it's unlikely that can continue for long.

Unhappy and unmotivated team members are far less likely to be productive, thus not able to implement changes as quickly, keep code quality as high or have great ideas to improve things.  The very fact that "retrospectives" or similar concepts are encouraged in documented "Agile" methodologies makes this very obvious.  I find generally everyone kind of knows this, but the real problem lies in the fact it's difficult to address it.  I am sure you will agree it can be very difficult on any project to get time allocated to removing technical debt.  This is generally a cost to the client for no perceived benefit (if you can't explain the benefits effectively).  I am hoping a definition like the one described above can help people out with this.  Basically we need to explain "in order to keep you happy and thus keep this project agile long term, we need to spend some time making sure the team is happy".

Why do we care about being Agile?

OK so you have read down to here.. or skipped if you got bored.  Anyhow why do we care about this buzzword and all seem to want to aspire to what it seems to promise?

The purpose of being agile is not to win some sort of competition or think your Software Product is better than some other because it's "more agile".  There is no value in comparing "agile" across Products or Teams.  The only value in measuring the "agility" of a product is so that you can identify areas to improve it to make more stakeholders happier in a more timely fashion.  If you can do this over time for a long period, then you would see a far better outcome than if you let things go off the rails and people are not happy.

Another perspective forgetting about clients and money etc is that all stakeholders involved in a project are generally spending time that they could be doing something else - maybe performing some other role or spending time with their family.  We owe it to ourselves and each other to make sure working on software projects is an enjoyable experience so that we are not wasting our lives being unhappy at work!

Measuring Agility

By the definition above, in order to try and measure how agile a project is, you are going to have to rely on qualitative measures that involve people's opinions.  If anyone has a better way of doing this other than "survey's" then let me know!  Here is one relevant survey I found that may be a good starting point:

For the most part I am not really interested in "measuring" it, just knowing when things are starting to go off the rails so I can try to bring them back on.  That can usually be done by talking to people and identifying any potential issues.

Tuesday, April 8, 2014

Yet another blog post on how to validate email addresses

I have read a lot of blog posts and Stack Overflow answers over the years on "the best way to validate an email address".  This article uses the example of validating in ASP.NET MVC but the theories could apply to any code base.

What I have learnt is that like most things, there is no "best" way, it depends on your requirements.

So I thought I would outline my most common requirements and how I do it.

Usually my requirements for capturing an email address are:

  1. Let the user know if they have possibly not typed in an email address at all (E.g. maybe they accidentally put their name in the email field).  This should be done client side so it is fast.
  2. I want the email validation to be as "loose" as possible, not check every known variation.  However, I want my mail sending code to work.  I don't want to queue up an email task on a background process for it to not be able to deal with the email address supplied.
  3. Optionally (not always), I want to know that someone actually got the email
(NOTE: High Performance is not a requirement - this is for validating emails as a user enters them, not batch validating.  So I am not worried about overhead of Regex vs String.Contains or anything like that)

To meet requirement #1, Regex works.  Basically I just want to know they typed something then the @ symbol and then something else to prevent "accidental entry".  Nothing more than that.  Regex is nice because you can easily apply the same logic on the server and client side.  This means the check to make sure it is possibly an email address can happen in the browser without even hitting the server.  One .NET Regex pattern for checking something@something is: ".+\@.+$" (please let me know in the comments if there is a better way).

For requirement #2, I do not use Regex.  Sure there are Regex patterns that claim to validate email "as per the spec" - but I am sure most email clients don't support a lot of those addresses and who knows if everyone actually implements as per the spec.  At the end of the day, I want to use the mechanism that is going to send my email to validate my email now, so I know it won't fail validation later.  If you are using a third party service to send emails, see if they have a validation service you can call.  If you are just using the built in System.Net.Mail namespace, you can verify with the "MailAddress" class.  In ASP.NET MVC, this is an example of a Model class with the Regex, plus checking that "System.Net.Mail.MailAddress" thinks that the address is valid:

For requirement #3, you could in theory try to check for a "bounce" - but that will not work in all scenarios.  Read receipts are also not reliable as the user has the option to not send one and not all mail clients support them.  If you really need to know if someone got the email, the only way I can think of is to put a link in there that they have to click to let you know.  They are not always going to click it either, so they might need some incentive.  A common example is when someone signs up to a service, in the email you put "click here to activate your account".  If they never click it, their account is never enabled.

Hopefully if you have the same requirements as this then these ideas will help you to quickly implement your email validation without researching options for hours and testing Regex patterns!

Tuesday, January 21, 2014

Code Coverage with OpenCover - example with NUnit (can run in TeamCity also)

I am a fan of code coverage as long as you aim for 100% coverage "of code that should be tested".  I don't think you will get some of the benefits of code coverage if you just turn it on for all code and say "let's aim for 80%".  A good code coverage tool should allow you to specify what to include or ignore by namespace, class name and even attributes (such as the "GeneratedCode" attribute).

OpenCover is a nice code coverage tool as it meets these needs and it's Open Source.  Later in this post I will go over how to set it up just using batch files to keep it nice and simple.

I use code coverage to set myself goals so I know what I am aiming to test in a certain time period.  For example, I could say "by the end of the week I want to have all my MVC Controllers tested" - so I add a filter to include the controllers in the code coverage report.  So I use it as a way of "defining success" while writing tests.

Keep in mind that having 100% coverage on some code does not mean you have tested all scenarios!  All it does is prove you have tested one scenario down that path.  So I use it as an indicator of whether something has been tested, not a measure to say the code works in every scenario.  But that is a lot better than nothing and accidentally missing some code!

Example implementation - Local Development Machine

Sample Application

To test the steps I am documenting here, I have created a very simple sample application.  If you want to play with this before adding it to your project or just see how it works, you can download the sample application here: https://github.com/nootn/OpenCoverExample.

Basic Setup with NUnit

For these instructions, I assume you have a Test project with some NUnit tests already succeeding.

  1. Install the following NuGet packages (it does not matter which project you install them in as they go to the solution level packages):
    1. NUnit.Runners
    2. OpenCover
    3. ReportGenerator
      *Take note of the version numbers of each of these packages that were installed as you need them later*
  2. Create a batch file in the root folder of your Test project called "_RunCodeCoverageInOutput.bat"
  3. In the contents, add the following (replacing 'ProjToTest'; with your project's name, 'TestProj' with the name of your test project, and changing the version numbers to match that of the packages you downloaded:

    "..\..\..\packages\OpenCover.4.5.2316\OpenCover.Console.exe" -target:"..\..\..\packages\NUnit.Runners.2.6.3\tools\nunit-console.exe" -targetargs:"/nologo TestProj.dll /noshadow" -filter:"+[ProjToTest]ProjToTest*" -excludebyattribute:"System.CodeDom.Compiler.GeneratedCodeAttribute" -register:user -output:"_CodeCoverageResult.xml"

    "..\..\..\packages\ReportGenerator.\ReportGenerator.exe" "-reports:_CodeCoverageResult.xml" "-targetdir:_CodeCoverageReport"
  4. Go to the properties of the batch file in Visual Studio and change the value of "Copy To Output Directory" to "Copy if newer"
  5. Build your solution, go to the output directory (E.g. bin\debug\) and run the batch file "_RunCodeCoverageInOutput.bat"

    1. If it doesn't work - check the console output to diagnose the error - feel free to comment on this post and I will try to help, or refer to the documentation
    2. If it does work, you should have a subdirectory created called "_CodeCoverageReport" and in there, open up "index.htm" which shows your nicely generated code coverage report!
      Code Coverage Report Example
      You can drill into each of the items on the left to see which lines of code have/have not been run

Include or Exclude more

To include or exclude more namespaces, you just alter the "filter" value.  The example filter I have above is "+[ProjToTest]ProjToTest*" which means "assembly = ProjToTest, namespace = ProjToTest".  If for example you had a particular namespace within that you wanted to exclude, you could add a "-" to the filter after the "+", E.g: "+[ProjToTest]ProjToTest* -[ProjToTest]ProjToTest.SomeNamespace*".  This illustrates you can be very granular by only including certain namespaces, or just include all then exclude specific ones - choose which ever option saves you time and makes sense.

There can be cases where auto-generated code ends up in namespaces you want to include, but you don't want to have the generated code included.  An example of this is T4MVC which generates code for your MVC project to create strongly typed references where you usually need to use magic strings.  One way I found around this which is included in the batch file above is to use the "excludebyattribute" switch.  You will notice above I have put -excludebyattribute:"System.CodeDom.Compiler.GeneratedCodeAttribute" which works if the generated code has that attribute (as T4MVC does).

For a more complicated example, see the code coverage I have created for the DotNetAppStarterKit sample MVC app: https://github.com/nootn/DotNetAppStarterKit/blob/master/DotNetAppStarterKit.SampleMvc.UnitTests/_RunCodeCoverageInOutput.bat.

Running in TeamCity with No TeamCity Changes

I couldn't find a TeamCity add-on for OpenCover (there was one for PartCover built in) and the only instructions I could find to get it working involved installing things on the server.  So that was another good reason to get this working as a batch file (so it can run in TeamCity self-contained within it's own source code).

This is how I got it working (using TeamCity version 7 at the time of writing) with the Code Coverage report showing as an "Artifact" so I can just click the link in TeamCity to view it:

  1. Create another batch file similar to "_RunCodeCoverageInOutput.bat" above called "_RunCodeCoverageInTeamCity.bat", but remove the "..\..\..\" in front of any executables, because TeamCity will run it from the root level, not have to go up 3 levels (you might have to play with this depending how TeamCity is configured for you)
  2. Again make the batch file "Copy if newer" as above
  3. In TeamCity, make sure your Artifact Path includes "_CodeCoverageReport":
    Code Coverage Team City Artifact Path

  4. Make a new build step after your solution is built to run the code coverage batch file.  It can just be a "Command Line" runner type and it just needs to run the batch file you created:
Open Cover Team City Build Step

Hopefully this helps you get started on your Code Coverage journey!

Monday, March 25, 2013

Command Query Separation to better manage Async and Caching for Performance

I have done a fair bit of reading and experimenting with a concept people call "CQS" or "CQRS". For a good discussion about the possible difference see Greg Young's post. From here on in I am going to assume they are the same thing for simplicity.

I am not going to do an in depth post about why you should or should not use this pattern/concept, I will only provide an overview. I am however going to show you one common application where I have found it brings great benefits using an ASP.NET MVC Web Application which relies heavily on server side caching of data and async controllers.

Tuesday, January 15, 2013

Time management tip - "Getting things done" method

I was speaking to a couple of people recently about a book I read recently called "Getting things done" by David Allen.

It is a very good read but basically it's a "program" you follow in order to get all your "todo" lists into one place and how to manage your tasks.

Run PowerShell from .NET program without System.Management.Automation

I have run into the problem a couple of times where it's not "easy" to add a reference to System.Management.Automation in order to execute PowerShell scripts from .NET code.  I found a way around it which works for simple scenarios where you might just want to execute some script and read the standard output plus error output.

Monday, January 14, 2013

Side Project: Website for Milliner

I built a web presence for a client who creates custom millinery, mostly headwear and some jewellery.  She had a business card and a facebook page but that was it.