Saturday 30 January 2016

Principles of Microservices



Model Around Business Concept
  • More stable than interface structured around technical concepts.
  • Reflect change in business process easily.
  • Use bounded contexts to define potential domain bounderies.
Culture of Automation
  • Keep the added complexity of microservices under control.
  • Automated testing is essential.
  • Deploy the same way everywhere.
  • Use environment definitions.
  • Creation of fully automated immutable servers.
Hide Internal Implementation Details
  • Technology agnostic API such as REST.
  • Services hide and don't share their respective database.
  • Use data pumps or event data pumps to consolidate data for reports.
Independently Deployable
  • Coexist versionned endpoints: consumers decide when they update themselves.
  • One service per host: eliminate impacts on unrelated service.
  • Blue/green or canary release: reduce risk of wrong release.
  • Consumer-driven contracts: catch breaking changes before they happen.
Isolate Failure
  • Don't treat remote call like local calls.
  • Plan for failure: Expect that failure will occur anywhere and everywhere.
  • Use appropriate timeouts.
  • Bulkheads and circuit breakers: limit the fallout of a failing components.
  • Sacrifice availability or consistency when network partition is unavailable.
Highly Observable
  • Logs aggregation with drill down capabilities.
  • Stats aggregation with drill down capabilities.
  • Use correlation IDS to trace calls through the system.
  • Use semantic monitoring with syntetic transactions injection to simulate real-user behaviors.
Decentralize All The Things
Organization: delegate decision making and control to the team.
  • Self-service: develop, test and deploy without relying on other teams.
  • Team own their services and are the domain expert.
  • Align team to the organization. (Conway's law).
  • Internal open source: Use open-source method to run projects.
  • Shared governance: people from each service team collectively share responsability for the technical vision of the overall system. 
Architecture: keep associated logic and data within service boundaries.
  • Avoid enterprise service bus, orchestration systems, dumb services.
  • Prefer choreography over orchestration.
  • Prefer dumb middleware with smart endpoints. 

Reference: Newman, Sam. Building Microservice. O'Reilly, 2015.

Friday 29 January 2016

CAP Theorem



Based on three trade-offs:
  1. Consistency: You get the same response if you go to multiple nodes.
  2. Availability: You get an answer for every request.
  3. Partition tolerance: System continue to process data even if a subsystem is unavailable.

In a distributed system, we can only have two of them.

A-P System. Always available, eventually consistent.
  • Even without a network failure, data replication between database node is not instantaneous.
  • At some point in the future, all nodes will see the updated data.
  • User always receive a response but may contain old data.
  • Usually the best and easiest option to build and scale.
C-P System. Always consistent, not always available.
  • Getting multinode consistency is very hard: use data store or lock service that offer these characteristics. Don't build it yourself.
  • When base nodes can't talk to each other, we cannot ensure consistency so we refuse to respond to the request (no availability).
C-A System.
  • CA system don't exist in distributed system. If your system have no partition tolerance, it's a single process run locally and can't be divided over a network.
You may have some AP capabillities and some CP capabilities mixed in the same system. This is usually happening with system build around micro-services.


Monday 9 November 2015

Takt Time


Average time between the start of production of one unit and the start of production of the next unit, when these production starts are set to match the rate of customer demand.

Not frequently used in software development.



Takt time = Working time available (per hour, day...) / Customer request (per hour, day...)

production rate > takt time = waste of inventory  / too much WIP.
production rate < takt time = waste of delay.
cycle time > takt time = bottleneck.



Example:


The customer need 3754 pencils per month.
Working hours = 20 workings day in a month * office hours (8) = 160 hours.

Takt time = 160 hours / 3754 = 0.043 hours = 2.6 minute.
Meaning that every 2.6 minute, the factory must start producing a new pencil.

Push vs Pull System


Characteristics

  • A station produce the most possible.
  • Produce no matter how much can be consume downstream.
  • Produce by batch.
  • Produce large lots. 
  • Forecast the demands from customer.
  • Employees perform the same tasks.

Benefits

  • In some particular situation, can reduce waste of transport.
  • Less prone to starving from shortage.
  • Simple to implement.  

Weakness

  • Forecasts are often inaccurate.
  • High level of WIP.
  • Generate waste of over production, delay and inventory.
  • Lead time is higher.
  • Encourage the silos mentality. 



Characteristics

  • A station produce based on the downstream consumer demand.
  • Overall production based on the customer demand.
  • Small or individual piece lot sizes.
  • If no demand, it is not made.

Benefits

  • Lead time reduced. (cost reduce, customer satisfaction increase)
  • Employees with multiple skills are used more efficiently (natural balancing).
  • Minimizes storage space needed. 
  • Low WIP.
  • Level of waste reduced.
Weakness
  • Need a system to control the level of WIP.
  • React poorly to high defect rate.
  • Suppliers shortage may starve quickly the chain of production.
  • High fructuation of demand may require the use of buffers.

Thursday 26 February 2015

Git

I recently started using git and tried to draw the command flow.
Let me know if you see any mistakes.





Wednesday 18 February 2015

Kanban System Deployed


Published with the authorization of Expretio technology


The following article is an example of a Kanban System that I designed for a software development company. Each organisation workflow have their own particularities so this should not be seen as a guideline.


Obeya room

A war room was dedicated to project visualization, architecture diagrams and daily meetings. One side of the room contain an heijunka board covering an entire wall, a small ressource allocation board and a digital heijunka board on touch screen.



A standup daily meeting is held every day at the same time to update all the boards. The room act as a converging point for strategic planning. It's map the entire team works and should bring an understanding of our system immediately and intuitively. 

Used by people of different roles, the room dissolve division between the department of the organisation.


Heijunka Board

The goal of the heijunka board is to visualize our entire value chain, from idea or concept to software in production. It's also make policies explicit and give a quick feedback based on objective data that allow us to track and optimize the flow.

The board give three essential informations:

  • what to produce
  • when to produce
  • how much to produce


 
The board itself is built from two magnetic whiteboard steel panels for a total of 10 feets long by almost 8 feets high. The lines and labelling are done from vinyl tape and stickers. No writing is necessary, the board is entirely managed with magnetic accessories for clarity and efficiency.

It is divided horizontally by stages or columns and vertically by swim lanes. Kanban card flow inside swim lanes through all the stages until completion.


Stages

The different stages of our workflow are shown as columns on the board. They are the steps required to bring work items from concept to working software.



A stages contain the following sections:

  • Criteria: determined what need to be met in order to move a work item to the next stage.
  • WIP limit: the maximum number of items in the stage. This include the items in both queues. Work in process limit is a fundamental aspect of Kanban. Further reading may be required to fully grasp the idea.
  • In progress queue: contain the item currently targeted by team members and not yet completed.
  • Done queue: contain the completed items that are waiting to be pull to the next stages. To be part of the done queue, a work item must respect all the criteria.
Swim lanes

Swim lanes can be used to split the board vertically so different type of work or project can flow independently. Each lane may have different policies regarding WIP limits and criteria. We don't mix the metrics between lanes: each swim lane record their own cycle and lead time.

Apart giving an overview of the activities of various project in parallel, having multi lanes on the same board also give an interesting feedback when resources are reallocated to different project or product.

Let say you move 2 team members from the development to the research lane, you will be able to probably observe a direct impact on the both lane simultaneously with some lead time gained on the development lane and some lead time reduction on the research lane.




It's a common practice to have a Urgent / Expedite lane for critical item. This particular lane have usually a WIP limit of 1 and should not be used frequently. Each item flowing in this lane affect the lead time of other lanes and can be a signal for some quality problem so we always keep an eye on it. 


Kanban cards


 
The kanban cards represent work orders. They are laminated 5” x 8” index cards and are reused after reaching completion. Formulated by following the user story convention, they are detailed in the project tracking software.

On the card itself, we only write a short description under the ticket number. When a team member is assigned to a card, a round magnet used as the person avatar is placed beside the card.

We use the following colors to qualify and distinguish the various class of work order:

  • Red: Rework item. More this waste appear frequently on the board, greater is the chance you got serious quality problems.
  • Blue: Valuable item for your customer that bring revenue to your organisation. Should be the most common item on the board.
  • Yellow: Non functionnal item. Improve the infrastructure by decreasing the lead time of the blue card and/or decreasing the frequency of red card.
  • Purple: Kaizen event. As it flow in the lane, it's trigger a team workshop event in each stage. Environment improvement and productivity optimization of the targeted stage is the goal of the event. A kaizen card always appear when 15 cards have been completed.
 
Tasks cards

 
When a kanban card reach a specific stage, it may be necessary to break it down in smaller task so team members can individually target portion of the work to be done. 

In our workflow, the tasks are defined in the TASK stage so when the work order reach the CODING stage, the tasks are ready to be targeted by programmers.

The task need to refer to an explicit and well defined area. In our case, tasks refer to architecture layers and are represented by small laminated colored card. Instead of writing the layer description on the card, we used again a color convention. By example, the persistence layer is pink and the presentation layer is green.

When a team member target a task, we put his avatar on the card. Once all the tasks are completed, we move the kanban card to the done queue.

 
Digital Heijunka Board

Digital board contain the same stages as the physical board. In our case, we were using a JIRA kanban board on a 40 inches touch screen.

After updating the physical board, we replicate the change by dragging the items on the digital board.

Using both physical and digital versions of the board and doing a little double bookeeping may first appear as a form of overhead but it's well worth the investment. 

Digital board is useful for generating various report on the lead and cycle time without effort. Cumulative flow diagram is a particulary expensive report to produce without a digital board. It is also practical for allowing off-site team member to collaborate.

You may be tempted to use only the digital board but it's usually a bad idea. A physical board is a better information radiator. It's always available, bigger, more visible to the entire organisation and easier to update.




Ressource Allocation Board

A ressource allocation board is installed beside the Kanban Board.




It's main function is to give an overview of the resource or team members that are not currently working or assigned to a production stage. The board contain categories such as vacation, sick leave, corporate event and client meeting.

We also use calendar date magnet with the resource in the vacation category so we know with a peek when someone should come back to work.

It's can be interesting to record metric for this board as well. By example, you may determine an average of corporate event per week, vacation leave for each month, etc. These trends may then help you for future planning and kaizen event session but be aware of bad corrolation before making conclusion.

A section of the board is reserved for storing magnetic accessories and unused kanban cards.



Tuesday 30 December 2014

More Jargon

"Fragile"

 - A software project using traditional "waterfall" lifecycle but called "agile" by the project managers.

"Fear Driven Development"

- When project management adds more pressure (fires someone, moves deadlines forward, substract or add resources to the project, etc).


http://blog.codinghorror.com/new-programming-jargon/

Thursday 1 May 2014

Kanban System





kan = visual
ban = card

Invented at Toyota as a scheduling system for just-in-time manufacturing.

Evolved to system that track work in process and optimize workflow

  • what to produce
  • when to produce
  • how much to produce

Kanban system combine practices of Lean Thinking and Theory of Constraint

Goal

Make the work flow fast through the whole value chain, from idea or concept to software in production.

Principles
  • Visualize
  • Limit WIP
  • Manage continuous flow
  • Make process policies explicit: based on objective data
  • Implement feedback loop: operation review
  • Improve collaboratively, evolve experimentally:
    - using models and the scientific method
    - can include Theory of Constraints model
    - can include Lean methods

Visual system

Visualize the flow

WIP in knowledge work isn't visual and this is the driving force behind wanting to visualize work.
  • Single point of access for the organisation
  • Map of our works
  • Bring understanding of our system immediately and intuitively
  • Explicit about:
    • roles
    • responsibilities
    • policies
    • WIP
    • rate of completion
    • structure of our process
    • impediments.
  • Dissolve division between the department of the organisation





Time Boxing







  1. Only 3 locks are available
  2. Timeboxing leave scope unlocked and adjust it accordingly

Feedback





  • Should be inexpensive to get (not generating waste).
  • The more quickly you get feedback, the more quickly you can remove defects and change a bad process into a slighter better one.
  • Creator of knowledge.
  • What work?
  • What should you change?
  • What shouldn't you change?

Types
  1. Product feedback
  2. Workflow feedback

Workflow feedback

  • Essential part of any process.
  • Continuous improvement is not possible without a proper level of feedback
  • It tell you about the quality of the workflow
Inadequate indicators and metrics can give distorted feedback thus aggravating a process.


Product feedback
  • It tell you about the quality of the product
  • Tests generate feedback
  • Prolonged feedback loop can generate more work
  • undetected defect degrade system integrity:



The vicious circle

Higher WIP generate noise and causes feedback to become slower and slower. Slow feedback generate delay and cause even more work to be created, which cause higher WIP, which prolongs the feedback...