In an earlier blog article I began outlining our new business strategy. That article talked about what we want. This will talk about how to get it.
The Sex Pistols sang: "Don't know want I want but I know how to get it..."
There is a lot of buzz around agile development practices, and I'm all for them. It's getting the client interested in them that seems to be the tricky part.
Here is a proposition: any time spent writing specs is time NOT spent developing, and hence it is time that is NOT getting a return on the client's investment. So (if we accept the proposition) the strategy is to minimise the amount of spec writing.
I've been down that route and it can end in tears. The problems are:
no specs = no idea what's being built for both client and developer
no specs = no limit to the scope
no specs = blown budget and time
The trick is to work out the *minimum* amount of specs that will mitigate the risk.
A quick aside: I was involved in a project recently that did not go quite to plan, and one of the main causes was the client had no idea of their business process. (They thought they did and we thought they did, but they really did not.)
Back to the main story: we're now doing a 3-part process for all new projects. The client pays up-front for the preparation of three documents, which is one day's work:
1) Project Proposal
2) Statement of Work
3) Cost Estimate
The Project Proposal contains the analysis of the business process and its aim is to make sure we all agree on what the project is about. Defining the business process up front has the effect of clarifying the client's understanding of what they do and how they do it, and how the solution will fit in and support their business. It specifies what is in the scope of the project and what is out of scope. This curtails a lot of confusion right up front.
The Statement of Work is a paper prototype of the proposed solution: mocked-up screen shots, print layouts, and a run-through of the database interacting with the business process. Creating this sorts out the data structure and the interface, and it forms the check list for the deliverables.
The Cost Estimate is the time and money to build the solution in the SoW. It clearly states that this is the first version of the solution and it's not the last, therefore changes to the project are not to be entertained at this point in time.
The time allocation is 3 hours to do the business analysis and 1 hour to prepare the Project proposal; then 3 hours to do the Statement of work and 1 hour for the Cost Estimate. Yes this is cutting it really fine but that's part of the discipline: this is the first stage of an iterative agile development process and we need to keep to short and quick. There is no time to debate for 15 minutes on whether to call the field "Surname" or "Last Name". Get it up and work it out later.
Similarly for the developers who build the database, the cost estimate will be tight so the scope and specs have to be followed closely. No cutting corners and no extra work.
Part of the success of this will be the use of a template to speed development. That's another story.
We have one client that has been through this methodology and another is coming up soon. SO far it's looking good.
Monday, October 4, 2010
Wednesday, September 29, 2010
Link to Linked-in
I've just updated my Linked-in profile and hooked up this blog there.
These blog posts are now appearing in Linked-in. Yeah!
These blog posts are now appearing in Linked-in. Yeah!
Monday, September 13, 2010
Business Re-think Part 1
I'm currently working to define and refine our marketing and business strategy. The executive summary is that we need to choose the clients we want to do business with, and we need to better define the way we want to do business with them. (The corollary from this is that we need to get our own house in order too.)
Our ideal client looks like this:
* The client already knows FileMaker and the benefits it can bring their business. We don't want to be spending time and money convincing them that FMP is a suitable development environment. We also don't want to be fighting the IT departments in larger organisations to implement something new and unknown to them.
* The client has the resources to complete the project. This is not just money, it includes the time and effort required to get involved with the development process.
* The client understands that the development process is iterative; that is, it will progress in stages and it won't all be finished in one go.
The way we want to do business is:
* Produce solutions that exceed the client's expectations, are good value for money and give them a good return on their investment.
* Get paid promptly for the work that we do.
* Have efficient and effective internal development processes that minimise "waste" and keeps the costs down for the client.
* We get interesting clients and interesting work so that our development team is kept busy and happy.
Next post I'll discuss the challenges and solutions as I see them to implementing successful projects and making the transition to where we want to be.
Our ideal client looks like this:
* The client already knows FileMaker and the benefits it can bring their business. We don't want to be spending time and money convincing them that FMP is a suitable development environment. We also don't want to be fighting the IT departments in larger organisations to implement something new and unknown to them.
* The client has the resources to complete the project. This is not just money, it includes the time and effort required to get involved with the development process.
* The client understands that the development process is iterative; that is, it will progress in stages and it won't all be finished in one go.
The way we want to do business is:
* Produce solutions that exceed the client's expectations, are good value for money and give them a good return on their investment.
* Get paid promptly for the work that we do.
* Have efficient and effective internal development processes that minimise "waste" and keeps the costs down for the client.
* We get interesting clients and interesting work so that our development team is kept busy and happy.
Next post I'll discuss the challenges and solutions as I see them to implementing successful projects and making the transition to where we want to be.
Monday, August 30, 2010
Project Management
One of the "essential" requirements for a developer position I was recently recruiting for was "project management". All of the applicants brought in for interview were extremely competent developers, but NONE of them had even the faintest idea what project management was. But they all thought they did.
They confused project management with time management. "Yes I do project management, I manage my time very carefully..." One applicant even boldly declared that project management was an utter, utter waste of time and we need just do everything *exactly* as he prescribes and NOTHING will go wrong. (I have since named this the MWoth project management methodology: my way or the highway.)
I was not surprised by this. The only reason I know anything about project management is because my wife is one of the best IT PMs in Sydney. Project management isn't something that you pick up doing something like development. It's a completely separate discipline.
I have for several months being thinking about a good analogy to describe all this, and one finally came to me last week while I was shopping with my mother. The analogy was the shopping itself.
My mother lives in Gosford but she was staying with us in Sydney, so we went to a supermarket she is not familiar with. As always she had her shopping list, and we started off with the vegetables because she always starts with the vegies.
We started at the first item on the list, the potatoes. So we went quickly and efficiently from the front of the store to the potato section. Next on the list was the tomatoes, so we went quickly and efficiently from the potatoes to the tomatoes. Then to the lettuce, then broccoli, and so on through the list.
From our perspective we were efficient and effective. We went the shortest route from where we were to the next supplies and wasted no time in making our selections. Our processes could not have been optimised further.
However to an outside observer we were - literally - all over the shop. The list of items was written in no particular order so we crossed the shop several times. I realised what was happening when, after getting the potatoes at one end, we went to the lettuce section at the other, then back for some pumpkin which was right next to the potatoes.
Huh? What does this have to do with developers and project management?
The problem was that planning had been put into what to get, but no effort had been put into thinking about an efficient way of doing the shopping itself.
The "getting of the items" is the development tasks, and developers work efficiently and effectively through the list of work an item at a time. Their job is to solve the problem at hand, the item next on the list. However, there are major efficiencies that can be gained from other things like optimising the order of the list that are outside the realm of "development" and it's these things that a project manager is concerned with. Many developers I work with have no interest in doing anything other than solving the problem at hand: some developers actively resist thinking any bigger than the task immediately at hand.
Getting the job done involves time management. If there is more than one job then there are efficiencies to be gained by planning the way the different jobs are done. This is project management. Project Management is not straight forward and it requires great and subtle skills and expertise.
Back to the example of shopping with my mum, the most efficiency would have been gained by:
1) getting the list of shopping items from mum;
2) re-arranging them so that they could be bought with the minimum time and effort (think travelling salesman);
3) giving the optimised list back to mum before she went shopping.
Step 2 is interesting because optimising the shopping list requires a detailed knowledge of the supermarket and the arrangement of the items on the floor. It's not enough to simply do all the vegies first; they also have to be optimised within themselves. It's subtle because each project is different: an shopping list that is optimised for one supermarket will probably not be optimal in another because the floor plan and layout of shelves will be different in each store.
Step 3 is version control and communication. The PM might have had an optimised list but perhaps it was never given to the shoppers, or the shoppers had an older version and worked to that. Or they decided to work without the list and improvise because they did not understand what all this rubbish about project management is.
In the end a developer was hired despite the fact they had no project management experience. I personally don't think developers (or other "doing" people) can do project management; my wife assures me that it's not appropriate for the a developer to be project managing their own work because it's a conflict of interest, and she'd know.
However it'd be nice if they could at least understand the difference between project management and time management, and drop a buzz-word like Prince2 into the conversation to at least sound as though they knew something about it.
They confused project management with time management. "Yes I do project management, I manage my time very carefully..." One applicant even boldly declared that project management was an utter, utter waste of time and we need just do everything *exactly* as he prescribes and NOTHING will go wrong. (I have since named this the MWoth project management methodology: my way or the highway.)
I was not surprised by this. The only reason I know anything about project management is because my wife is one of the best IT PMs in Sydney. Project management isn't something that you pick up doing something like development. It's a completely separate discipline.
I have for several months being thinking about a good analogy to describe all this, and one finally came to me last week while I was shopping with my mother. The analogy was the shopping itself.
My mother lives in Gosford but she was staying with us in Sydney, so we went to a supermarket she is not familiar with. As always she had her shopping list, and we started off with the vegetables because she always starts with the vegies.
We started at the first item on the list, the potatoes. So we went quickly and efficiently from the front of the store to the potato section. Next on the list was the tomatoes, so we went quickly and efficiently from the potatoes to the tomatoes. Then to the lettuce, then broccoli, and so on through the list.
From our perspective we were efficient and effective. We went the shortest route from where we were to the next supplies and wasted no time in making our selections. Our processes could not have been optimised further.
However to an outside observer we were - literally - all over the shop. The list of items was written in no particular order so we crossed the shop several times. I realised what was happening when, after getting the potatoes at one end, we went to the lettuce section at the other, then back for some pumpkin which was right next to the potatoes.
Huh? What does this have to do with developers and project management?
The problem was that planning had been put into what to get, but no effort had been put into thinking about an efficient way of doing the shopping itself.
The "getting of the items" is the development tasks, and developers work efficiently and effectively through the list of work an item at a time. Their job is to solve the problem at hand, the item next on the list. However, there are major efficiencies that can be gained from other things like optimising the order of the list that are outside the realm of "development" and it's these things that a project manager is concerned with. Many developers I work with have no interest in doing anything other than solving the problem at hand: some developers actively resist thinking any bigger than the task immediately at hand.
Getting the job done involves time management. If there is more than one job then there are efficiencies to be gained by planning the way the different jobs are done. This is project management. Project Management is not straight forward and it requires great and subtle skills and expertise.
Back to the example of shopping with my mum, the most efficiency would have been gained by:
1) getting the list of shopping items from mum;
2) re-arranging them so that they could be bought with the minimum time and effort (think travelling salesman);
3) giving the optimised list back to mum before she went shopping.
Step 2 is interesting because optimising the shopping list requires a detailed knowledge of the supermarket and the arrangement of the items on the floor. It's not enough to simply do all the vegies first; they also have to be optimised within themselves. It's subtle because each project is different: an shopping list that is optimised for one supermarket will probably not be optimal in another because the floor plan and layout of shelves will be different in each store.
Step 3 is version control and communication. The PM might have had an optimised list but perhaps it was never given to the shoppers, or the shoppers had an older version and worked to that. Or they decided to work without the list and improvise because they did not understand what all this rubbish about project management is.
In the end a developer was hired despite the fact they had no project management experience. I personally don't think developers (or other "doing" people) can do project management; my wife assures me that it's not appropriate for the a developer to be project managing their own work because it's a conflict of interest, and she'd know.
However it'd be nice if they could at least understand the difference between project management and time management, and drop a buzz-word like Prince2 into the conversation to at least sound as though they knew something about it.
Saturday, August 21, 2010
Design-O-Matic
While doing the work for the new template interface design I put together a little database to play around with the possible combinations of colour fills and gradients. (It was also an opportunity to see what could be done with conditional formatting.)
When run in Windows the gradient images display a "smearing" on the right hand side, apparently due to the way Windows enlarges images. The solution is to not expand the images, but to create them the correct size or oversize and shrink them. Since this was a quick hack for me to play with all the options so I'll leave it as it is.
The result is available for your personal enjoyment. This file requires FMP 10.0 or later. For any commercial use please contact me. The solution is provided as-is with no warranty, use at your own risk, ymmv, etc.
When run in Windows the gradient images display a "smearing" on the right hand side, apparently due to the way Windows enlarges images. The solution is to not expand the images, but to create them the correct size or oversize and shrink them. Since this was a quick hack for me to play with all the options so I'll leave it as it is.
The result is available for your personal enjoyment. This file requires FMP 10.0 or later. For any commercial use please contact me. The solution is provided as-is with no warranty, use at your own risk, ymmv, etc.
New Template Part 2
In April I mentioned I'm updating my standard starter template: this has been completed and used for a couple of solutions now.
One of the main reasons for the upgrade was the interface design changed introduced in FMP 10 and carried over to FMP 11: the status area is now a user-configurable toolbar area at the top of the screen, and not at the left hand side as had been in FMP 9 and earlier, since time began.
Computer monitors are now all "widescreen" so while they offer a broad vista horizontally, vertical screen space is at a premium. The new design is very frugal with screen space vertically to allow for this.
I have also put a HUGE amount of development effort into managing multiple windows. Many solutions use global fields or variables to track interface and navigational data such as the currently selected tab or the field that the table is sorted by. All of these solutions suffer the problem that they only work for one window: if the user makes a new window of the current screen, there is no way to track this navigation data for each window separately.
My solution stores the navigation and interface data in global variables that incorporate the window name into the variable, thereby allowing multiple windows to exist independently and simultaneously. The trick (or one of them, there are many) is to encode the window name to ensure that only variable-legal characters are used. Script triggers introduced in FMP 10 generate the global variables automatically as users create windows and clear them when the windows are closed.
To achieve the creation of the global variable names I create a 32 bit binary hash of the window name and convert this to decimal, which reduces it to around 16 characters (variable names can be up to 100 characters). This step alone required my learning about hash algorithms, selecting one that would be appropriate for the reasonably short window names expected - the HashFNV1a algorithm - and re-creating the algorithm as a FileMaker recursive custom function. This in turn required the development of a custom function to XOR two binary numbers. I've shared both my hash and XOR functions on Brian Dunning's custom function site, bringing my contribution to 12 functions there.
XORbin
HashFNV1a32
I then created custom functions to work as an "interface" for generating the variable names in one step, and another for reading the variable name in one step. The system worked extremely well until I had a large data set on a hosted file, when in list view the screen would slow to a crawl. The problem was traced to the fact that the hash was being recalculated for each record displayed on the screen. I solved it by "patching" the hash custom function with another that cached the window name and the hash result in a global variable, so the hash value is only ever generated once per session. Problem solved!
This system is now in 3 solutions that I have developed and implemented, and it's working well.
Interestingly, in a previous job I was in the interview panel for my replacement. They mentioned that that had used my custom functions for calculating the DistanceBetweenPoints in one of their biggest projects.
One of the main reasons for the upgrade was the interface design changed introduced in FMP 10 and carried over to FMP 11: the status area is now a user-configurable toolbar area at the top of the screen, and not at the left hand side as had been in FMP 9 and earlier, since time began.
Computer monitors are now all "widescreen" so while they offer a broad vista horizontally, vertical screen space is at a premium. The new design is very frugal with screen space vertically to allow for this.
I have also put a HUGE amount of development effort into managing multiple windows. Many solutions use global fields or variables to track interface and navigational data such as the currently selected tab or the field that the table is sorted by. All of these solutions suffer the problem that they only work for one window: if the user makes a new window of the current screen, there is no way to track this navigation data for each window separately.
My solution stores the navigation and interface data in global variables that incorporate the window name into the variable, thereby allowing multiple windows to exist independently and simultaneously. The trick (or one of them, there are many) is to encode the window name to ensure that only variable-legal characters are used. Script triggers introduced in FMP 10 generate the global variables automatically as users create windows and clear them when the windows are closed.
To achieve the creation of the global variable names I create a 32 bit binary hash of the window name and convert this to decimal, which reduces it to around 16 characters (variable names can be up to 100 characters). This step alone required my learning about hash algorithms, selecting one that would be appropriate for the reasonably short window names expected - the HashFNV1a algorithm - and re-creating the algorithm as a FileMaker recursive custom function. This in turn required the development of a custom function to XOR two binary numbers. I've shared both my hash and XOR functions on Brian Dunning's custom function site, bringing my contribution to 12 functions there.
XORbin
HashFNV1a32
I then created custom functions to work as an "interface" for generating the variable names in one step, and another for reading the variable name in one step. The system worked extremely well until I had a large data set on a hosted file, when in list view the screen would slow to a crawl. The problem was traced to the fact that the hash was being recalculated for each record displayed on the screen. I solved it by "patching" the hash custom function with another that cached the window name and the hash result in a global variable, so the hash value is only ever generated once per session. Problem solved!
This system is now in 3 solutions that I have developed and implemented, and it's working well.
Interestingly, in a previous job I was in the interview panel for my replacement. They mentioned that that had used my custom functions for calculating the DistanceBetweenPoints in one of their biggest projects.
Wednesday, August 18, 2010
The Right Way to implement a new system
Last week I completed an implementation for a client. This went to plan, unlike another database implementation I was involved in some time ago. Why did this one one go better? Because I learned from the previous implementation that there are three key areas that need to be addressed: data migration; functionality; and user acceptance.
Data Migration
This is usually the hardest part and is often the most disappointing because the existing data is uaually in the wrong “shape” and is poorly entered as well, and fixing it can take a huge amount of time and money and still be disappointing. Consequently anything that can be done to make data migration easier up front should be done. In this case the new database started with the old database so everything has the same field, table and relationship names to make data imports trivial. Junk fields, tables and relationahips were deleted. New fields were added to the new system *and* the old system where appropriate. In the old system the new fields used meticulously hand-crafted calculations to auto-magically clean up the data and calculate key values, so the old system is constantly migrating its own data even while it’s still in production. When absolutely necessary data is manually cleaned-up to sort out issues, and the old system’s interface was updated to prevent bad data from being entered again. All clean-up processes are scripted. These scripts are built-up and refined as the development proceeds and every day the migration process is re-run and checked. This reduced the data migration to a simple process that was completely automated and took less than 2 hours to run. All that was left was to manage the implementation of the new system and the de-commissioning of the old. That was something the previous migration got right: set up a development server that becomes the production server.
Functionality
The database has to do what people need and work as expected. The only way to achieve this is to scope and spec carefully up front, keep the development focussed, and to test, test and test again. It is vitally important to get the show-stoppers fixed the main process have to work end-to-end. Triage the other bugs and solve them based on priority (eg, the reporting isn’t working but that isn’t needed for 6 months, it will be ready then). And manage expectations: let the clients know that after migration there will be some minor issues, and that time has been allocated to fix them promptly. All the major issues should be sorted. Be careful not to delay shipping entirely: go when the system is usable, not finished.
User Acceptance
If the new system is easier to use than the old system the users will WANT to change. Work out where the pain points are in the old system and remove them in the new system. In this case the pain points were: an interface that was illegible (white text on a navy blue background) with an interface that was difficult to find information in; a developer-created user access control system that required manual log-in without providing any real security; a developer-built data entry system that caused data to be lost if the users closed the window without clicking the “save” button; a non-normalised data structure that required excessive data entry but made reporting difficult or impossible. The new system sported a new interface that was significantly similar to the old so the users could see some similarlty and transfer some skills from the old to the new. The faux-access system was removed and replaced with real accounts and access privileges, and external authentication was used to get single sign-on: no prompting for passwords at all. The “save” buttons were removed and data entry just works with no lost data, and the new data structure has reduced data entry time adn made the generation of reports significantly easier.
What could be improved
There is always something that can be done better and for this project it was communication and training.
The development lasted for a couple of months and while those immediately involved in the work were kept up to date, I made little effort to communicate with the other users of the system or the wider organisation in general. Both are important.
I realised this near the end and made sure that before the migration I personally met with or telephoned each user (some are interstate) and walked them through the new system and the basic database processes. This also doubled as the training session. The similarlity with the old system meant that most users were able to transfer to the new system reasonably easily.
I do think that the users could have had more training but the geographical spread made that difficult, and developing material and delivery takes time and money. A user guide has been developed but I don’t really think these are as useful as they could be: for one thing they go obsolete quickly as changes to the system are made, and I EXPECT to be making changes after implementation.
The client is happy and so am I!
Data Migration
This is usually the hardest part and is often the most disappointing because the existing data is uaually in the wrong “shape” and is poorly entered as well, and fixing it can take a huge amount of time and money and still be disappointing. Consequently anything that can be done to make data migration easier up front should be done. In this case the new database started with the old database so everything has the same field, table and relationship names to make data imports trivial. Junk fields, tables and relationahips were deleted. New fields were added to the new system *and* the old system where appropriate. In the old system the new fields used meticulously hand-crafted calculations to auto-magically clean up the data and calculate key values, so the old system is constantly migrating its own data even while it’s still in production. When absolutely necessary data is manually cleaned-up to sort out issues, and the old system’s interface was updated to prevent bad data from being entered again. All clean-up processes are scripted. These scripts are built-up and refined as the development proceeds and every day the migration process is re-run and checked. This reduced the data migration to a simple process that was completely automated and took less than 2 hours to run. All that was left was to manage the implementation of the new system and the de-commissioning of the old. That was something the previous migration got right: set up a development server that becomes the production server.
Functionality
The database has to do what people need and work as expected. The only way to achieve this is to scope and spec carefully up front, keep the development focussed, and to test, test and test again. It is vitally important to get the show-stoppers fixed the main process have to work end-to-end. Triage the other bugs and solve them based on priority (eg, the reporting isn’t working but that isn’t needed for 6 months, it will be ready then). And manage expectations: let the clients know that after migration there will be some minor issues, and that time has been allocated to fix them promptly. All the major issues should be sorted. Be careful not to delay shipping entirely: go when the system is usable, not finished.
User Acceptance
If the new system is easier to use than the old system the users will WANT to change. Work out where the pain points are in the old system and remove them in the new system. In this case the pain points were: an interface that was illegible (white text on a navy blue background) with an interface that was difficult to find information in; a developer-created user access control system that required manual log-in without providing any real security; a developer-built data entry system that caused data to be lost if the users closed the window without clicking the “save” button; a non-normalised data structure that required excessive data entry but made reporting difficult or impossible. The new system sported a new interface that was significantly similar to the old so the users could see some similarlty and transfer some skills from the old to the new. The faux-access system was removed and replaced with real accounts and access privileges, and external authentication was used to get single sign-on: no prompting for passwords at all. The “save” buttons were removed and data entry just works with no lost data, and the new data structure has reduced data entry time adn made the generation of reports significantly easier.
What could be improved
There is always something that can be done better and for this project it was communication and training.
The development lasted for a couple of months and while those immediately involved in the work were kept up to date, I made little effort to communicate with the other users of the system or the wider organisation in general. Both are important.
I realised this near the end and made sure that before the migration I personally met with or telephoned each user (some are interstate) and walked them through the new system and the basic database processes. This also doubled as the training session. The similarlity with the old system meant that most users were able to transfer to the new system reasonably easily.
I do think that the users could have had more training but the geographical spread made that difficult, and developing material and delivery takes time and money. A user guide has been developed but I don’t really think these are as useful as they could be: for one thing they go obsolete quickly as changes to the system are made, and I EXPECT to be making changes after implementation.
The client is happy and so am I!
Tuesday, April 13, 2010
Interface Inspiration: #1
I'm in the process of updating my basic FileMaker template. The current layout design is exceptionally functional and not too unsightly (IMHO) but I got a bit worried when a colleague described it as "retro Mac OS X 10.1". Ouch. True though.
I want the new template to be designed for FMP 10 and later, so the interface will assume the toolbar is at the top. I want it to be as "open" as possible to the user, and as close to the standard FileMaker interface as possible.
So I've been taking a look around the web and have discovered a couple of interesting samples. Note that I'm only looking at the interface design and not other functionality.
FM Starting Point is a completely open and free template from Richard Carter Consulting. I have no affiliation. Beautifully crafted and nice looking
Deskspace a commercial database for legal types. There are screen shots of the user interface
Seed Code these folks are legendary. They have a couple of screen shots here but notice that it's using FMP 9 with the status area to the left. The design of their calendar solution is excellent but not really what I'm after.
Next post I'll discuss my "philosophy" for the interface design and lay down some criteria.
Subscribe to:
Posts (Atom)