Hadoop Platform as a Service in the Cloud

http://techblog.netflix.com/2013/01/hadoop-platform-as-service-in-cloud.html

Posted in Uncategorized | Tagged , , , | Leave a comment

How Leaders Should Manage Failure

(Former President of India APJ Abdul Kalam at Wharton India Economic forum , Philadelphia , March 22,2008 )

Question: Could you give an example, from your own experience, of how leaders should manage failure?

Kalam: Let me tell you about my experience. In 1973 I became the project director of India’s satellite launch vehicle program, commonly called the SLV-3. Our goal was to put India’s “Rohini” satellite into orbit by 1980. I was given funds and human resources — but was told clearly that by 1980 we had to launch the satellite into space. Thousands of people worked together in scientific and technical teams towards that goal.

By 1979 — I think the month was August — we thought we were ready. As the project director, I went to the control center for the launch. At four minutes before the satellite launch, the computer began to go through the checklist of items that needed to be checked. One minute later, the computer program put the launch on hold; the display showed that some control components were not in order. My experts — I had four or five of them with me — told me not to worry; they had done their calculations and there was enough reserve fuel. So I bypassed the computer, switched to manual mode, and launched the rocket. In the first stage, everything worked fine. In the second stage, a problem developed. Instead of the satellite going into orbit, the whole rocket system plunged into the Bay of Bengal. It was a big failure.

That day, the chairman of the Indian Space Research Organization, Prof. Satish Dhawan, had called a press conference. The launch was at 7:00 am, and the press conference — where journalists from around the world were present — was at 7:45 am at ISRO’s satellite launch range in Sriharikota [in Andhra Pradesh in southern India]. Prof. Dhawan, the leader of the organization, conducted the press conference himself. He took responsibility for the failure — he said that the team had worked very hard, but that it needed more technological support. He assured the media that in another year, the team would definitely succeed. Now, I was the project director, and it was my failure, but instead, he took responsibility for the failure as chairman of the organization.

The next year, in July 1980, we tried again to launch the satellite — and this time we succeeded. The whole nation was jubilant. Again, there was a press conference. Prof. Dhawan called me aside and told me, “You conduct the press conference today.”

I learned a very important lesson that day. When failure occurred, the leader of the organization owned that failure. When success came, he gave it to his team. The best management lesson I have learned did not come to me from reading a book; it came from that experience.

(Complete interview here: http://knowledge.wharton.upenn.edu/india/article.cfm?articleid=4276)

Posted in Uncategorized | Leave a comment

BizTalk conference Stockholm videos

Authors of the Applied Architecture Patterns on the Microsoft Platform book presented a lecture series based on their book at BizTalk user group Sweden. These recordings are available at Channel9 now: http://channel9.msdn.com/Tags/biztalk+user+group+sweden

Posted in BizTalk | Leave a comment

You’ll Love The new Internet Explorer 9 Beta

Microsoft has launched a new browser – Internet Explorer 9 (IE9) beta. I know you’re thinking that it is just another browser, ain’t it? Yes, it is a new browser, but it is different. It is a BROWSER that is redefining web browsing!

IE9 beta provides a rich and immersive experience of your favorite websites that make online browsing much more beautiful! You don’t believe me? Even other browsers agree. Check what competitors have to say about IE9 beta:

http://www.ft.com/cms/s/2/7c1270c2-c029-11df-b77d-00144feab49a.html

I downloaded the IE9 beta and I am sharing three features that left me pleasantly surprised:

1. The multimedia experience in IE9 beta is AMAZING! It was like one of those sci-fi movies. Before IE9 beta, I didn’t even know a browser could do all that. Download IE9 beta and check out Amazon.com or IMDB.com and you’ll know what I mean!

 

2. With IE9 beta, the sites that I opened looked cool and aesthetic! They almost shine through! What I loved more was that with IE9 beta, I can go to my favorite sites/web pages straight from my desktop. So no more searching for URLs and navigating through numerous pages.

 

3. Everything on IE9 beta is fast and quick. IE9 beta takes full advantage of the power of the computer’s hardware to make the sites superfast! Almost as fast and responsive as applications installed on the computer.

 

I am super excited about how IE9 beta has enriched my online experience. Download the IE9 beta version today, to experience it for yourself.

Log on to www.beautyoftheweb.com

Posted in Microsoft | Leave a comment

दास्ताँ ऐ जायका

(seems so like my story…)
 

मुझे खाना बनाना पसंद है। वैसे सही कहा जाये तो खाना बनाना नहीं वरन खाना बनाते हुए प्रयोग करना पसंद है। एक दिन कहीं पढ़ा कि सब्जी में पड़ने वाले मसाले प्राकृतिक होने के कारण उन्हें पकाना नहीं चाहिए, सिर्फ ऊपर से डालना चाहिए। उस रात बनी सब्जी कूड़ेदान में दो दिन तक सड़ती रही। एक दिन गरम दूध में केले डाल दिए तो दूध फट गया। कभी दही, कभी नींबू, कभी अचार मसाला डाल कर सब्जी को ख़राब किया। अनुसंधान के नाम पर कई बार खाने में तला लगा।

बारिश के मौसम में चाय-पकोड़े खाने के लालच में पकोड़े तेल में जलाकर काले किये और लेमन टी बनाने के चक्कर में फटे दूध की चाय बनाई। दीवाली में पटाखे तो सभी फोड़ते हैं मगर मैंने तेल भरी कढ़ाई में समोसे फोड़े। दक्षिण भारतीय पाक शैली से प्रभावित हो एक रविवार डोसा मसाला पावडर खरीदा तो पूरी छुट्टी उस तवे को साफ़ करने में निकली, भूखे पेट रहे सो अलग। कभी ज्यादा पानी से चावल की खीर बन गयी तो कभी कम पानी की वजह से कुकर फटते – फटते बचा। ऑस्ट्रेलिया के मानचित्र का चिंतन करते हुए बनाई गयी रोटी अफ्रीका का अनुसरण करने लगी। कच्चे पपीते की सब्जी खाने के बाद एक हफ्ते पेट दर्द रहा। मगर हम भी हार मानने वालों में से कहाँ थे!

तय हुआ कि टीवी पर "कुकरी शो" देखे जाएँ। क्या पता चौपिंग पैड पर सर रगड़कर अपनी जड़मति भी सुजान हो जाए? मगर आजकल टीवी पर कुकरी कम और खुखरी शो ज्यादा लोकप्रिय हैं। जैसे तैसे संजीव कपूर के शो खाना खज़ाना को ढूँढा गया। एक पल के लिए लगा कि खाना बनाना कितना आसान है।

देखिये कितने सुन्दर चाइना की तश्तरियों में सलीके से छल्लेदार सब्जियां काट कर रखी हैं। छोटे छोटे प्यालों में जायफल, दालचीनी, इलाइची, जाफरानी केसर, चक्र फूल और न जाने कौन – कौन से अजूबे मोहक पदार्थ रखे हुए हैं। दूसरी तरफ चीनी मिटटी के सफ़ेद कटोरों में नमक, हल्दी, मिर्च, धनिया, हींग, अदरक और लहसुन भी आत्मसंतुष्टि से भरे हुए हैं। जैतून का तेल सुराही नुमा जग में बैठा बाकी सब को यूँ देख रहा है जैसी बाढ़ ग्रस्त इलाके को हेलीकाप्टर पर बैठा मंत्री। सारे बर्तन एकदम साफ़ सुथरे और चमकते हुए, जैसे १५ अगस्त की प्रभात फेरी के लिए तैयार स्कूल के बच्चे।

ऐसा लगता है जैसे बोरीवली में फिल्माए जा रहे एक सीन से कैमरा एक पल के लिए स्विट्ज़रलैंड की हसीन वादियों में पहुँच गया हो। सब कुछ इतना सरल, सहज, सुन्दर और सुगम। बस सब्जियां कटी हों, तेल गरम हो, मसाले पिसे हों – सब्जी डालो और पकाओ – न सब्जी काटने की समस्या, न बर्तन धोने का टेंशन! मन हुआ कि ऐसे ही किसी टीवी शो में भर्ती हुआ जाये, इसी बहाने रोज अच्छे पकवान खाने को मिलेंगे।

खैर लौट के बुद्धू घर को आये। समझ में आया कि यह अपने बस का रोग नहीं – जिस का काम उसी को साजे, और करे तो डंडा बाजे। उस दिन से आज तक अपनी पाक प्रतिभा मैगी बनाने तक ही सीमित है।

Posted in Hindi | Leave a comment

Learn How Google Works: in Gory Detail

How Google Works.

Infographic by PPC Blog

Posted in Computers and Internet | Leave a comment

Principles of Agile Software Development

  • Get case 1 fully working before starting case 2. Another way of saying this to use a kitchen metaphor is: “Serve the current meal before starting to cook the next“.  The biggest problem with software development is to start a bunch of things in parallel, because inevitably work will include something that is later discarded, meaning wasted work.  Work on one case; get it fully functional; get the tests running; write the documentation; check it all in as a finished piece of work, before you start on the next case.
  • Never break the build.  Pretty obvious, but must be included in any list of software development advice.  A programmer who is taking all the proper precautions to test before checking in will never break the build.  If the build is broken, it is always because someone took a shortcut.
  • Never implement a routine before it is needed in a use case.  When implementing a particular class, you should have a particular use case in mind, and you should implement only the methods required for that use case.  You might think about  the potential for other capabilities on a class, and you might document this in a comment, but implementation should wait until it is actually needed in a use case.
  • Never add a data member before it is needed in a use case. Exactly like above except with regard to class data members.  It may seem obvious that a “customer” record will need a “ship to address”, but that ship to address should not be implemented until you have a use case which specifically needs it.
  • Don’t be afraid to make a decision;  don’t be afraid to change an earlier decision.  Agile development is about responding to uncertainty and quickly responding.  Early in development you do not have complete information.  You should delay decisions as long as possible, but there comes a time that a decision is needed to move forward.  You cannot hold up decision until that information comes in.  Instead, make the best decision you can on the available information.  Later, when new information arrives, don’t be afraid to change that decision.  (Some dinosaurs call this flip-flopping, but I call it reacting to a changing environment.)
  • Continually learn how to improve quality. This job never ends, so you should expect to be constantly on the lookout for things that could be improved, and collect examples of ways that quality problems were identified and addressed.
  • Measure, measure, measure. Agile development is help address the problem of uncertainty about the future, but there should be no uncertainty about the past.  Tests should be continually running.  Performance of every run should be measured and recorded.
  • Design around people, not systems. Too often developers get sidetracked into designing for technical opportunities.  You should never lose sight of the ultimate purpose of the software, and that is to help people get work done.
  • Tests are part of the product.  Many developers and managers consider the product to be what you ship to the customer, and everything else less important.  The tests should be considered an actual part of the product, worthy of full consideration during design, and even, in many cases, delivered with the product to the customer. (This latter part is controversial, but a built-in self-test as part of a software delivery takes inconsequential space, and yet provide tremendous benefit when needed.  Such an approach should be considered.)
  • Write the test before the code. The test itself can be used to clarify the design for exactly what is needed.   Many times there are flaws in the design approach which are discovered when working through the test cases.  Think how much time would be saved to work through those cases before coding.  But: write the test for case1, and code for case 1, before starting case 2.
  • Eliminate Waste. Frankly, another ubiquitous platitude which must be included in any list of development principles because it is so important.  There is no end to the job of looking for waste where it exists and eliminating it.  Eliminate anything that does not add value to the actual customer.  If you cannot identify the value to the customer, then you probably don’t need it.
  • Build a culture of immediate response to build breakage. Understand that when the build is broken, it affect everyone in the project, and so there is nothing more important than making sure that the central core code is building and testing properly.  I have seen teams that allowed broken tests to persist for months because it was someone else’s job.  Everyone suffered, but nobody acted.  Instead, there needs to be widespread recognition that a little work will pay back in a big way over the team.
  • Every team members needs to understand the needs of the customer.  Large complex projects must be broken into separate teams and further divided for handing out to developers, but this should never be done to the extent that people lose sight of the desires and goals of the actual users of the final product.
  • Keep related definitions together. Structure the code so that highly related things are located together, possibly within one class.   This is a standard OO design principle of encapsulation.  Ideally, all the code outside the class will not need to know the details of the internal workings.  Some developers delight in spreading details across multiple files in order to organize in different way: such as to keep all the same data types together, or to organize alphabetically.  For example, putting all the constants in one class in a different package from where they are being used adds unnecessarily to the complexity of the program.   The guiding rule should be to group by relatedness with the result being to hide complexity.
  • Always run the tests before checking in. This guideline will help you satisfy the  “never break the build” guideline.
  • Premature optimization is the root of all evil. A quote from Don Knuth which rings true today.  Code should be written well to avoid needless waste at the micro level, but optimization beyond the individual method level should wait until testing within the entire program with a stress test bases on an actual end user use case.  Intuition of what is important for overall performance is almost always wrong when based only on a static understanding of the code.  Instead, measure the behavior of the complete system, to identify the 1% of the code that really makes a different in performance, and focus on that.
  • Minimize backlog of uncompleted coding tasks. When a developer starts to work on a use case, there is a cost associated with all the code that has been modified, but not completed and tested.  Holding uncompleted changes for days or weeks adds up to a significant risk of waste due to rework.  Consider three tasks estimated to take 1 day each.  Starting all three at one time, and working in parallel for three days involves an accumulation of 9 “units” of cost.  But doing each task sequentially, completing each task before starting the next, involves only 3 “units” of cost.  This is not intuitive.  Our intuition tells us that while we are in there, we might as well do three things at once, before buttoning the work up.  But software is not like physical construction.  Short, quick, and complete jobs not only cause less cognitive load, but also reduce the chance that uncompleted work will conflict with another person’s uncompleted work.
  • Never over generalize functionality. This is also known as “YAGNI – You Aren’t Going to Need It” .  While coding a particular class, programmers like to think with a small tweak this class might be used for several other purposes.  This is fine if those purposes are required by the current use case, but usually the programmer is thinking about uses which have not been invented yet, and which may in fact never be needed.  (This subject always reminds me of the classic Saturday Night Live skit about the product which was both a floor wax, and a dessert topping.)
  • Never use 3 lines when 2 lines would do. Succinctness in code pays every time someone else has to read it.  But don’t shrink the code to the point of being difficult to read.  Smaller, well written code can easier to maintain and easier to spot errors in, than verbose, ornately written code.  Always simplify as much as possible, but no more.
  • Never ever measure code by counting lines. The number of lines needed to do a particular task varies greatly from programmer to programmer, and from style to style.  The number of lines of code does not tell you much of anything about the completeness or the quality of the code.  Code quality can vary by a factor of 200, and this overwhelms any usefulness of the count of the lines.  Count instead functioning use cases.
  • Continually re-design and re-factor. Apply this cautiously because some code is brittle and difficult to change, but in general you should not be afraid to change the code to match the real use case.  A data member may have been an integer in the past, but when a use case requires it to be a floating point don’t be afraid to change it.
  • Delete dead code. There is a tendency to let “sleeping dogs lie” when it comes to large blocks of code that is not well understood.  One example is adding a new method to a class to replace another, quite often the developer will leave the old method there “just in case”.  Some effort should be expended to check to see if that method is needed, and delete it if there is no evidence that it is needed.  The worst offense is commenting out blocks of code, and leaving that commented code around.  Commented code should be deleted as soon as you know that the tests run, and certainly before checking it in.  It is easy to add code at any time, it is hard to delete code at any time.  Therefore, at a particular time that you have a good idea that something might not be needed, and small extra effort to verify this and eliminate the code will make the codebase more maintainable.
  • Don’t invent new languages. Programmers love to make text files that drive functionality in way configurable at run-time.  There are no end of configuration files to be able to change the behavior of the program without recompiling.  The advent of XML is driving an unending chain of specialized custom “scripting languages” that allow functionality to be “programmed” by the end user without having to compile.   The flaw in this reasoning is that the precise definition of the behavior of the operations almost never well defined outside of the context of a particular implementation, and these types of scripting languages are mainly useful only to people who have an intimate knowledge of the internal working of the code body in question.  Thus, real end users without detailed internal knowledge can never possibly know what is necessary to anticipate the effect of complex combinations of commands.   Scripting languages have a use, and cannot be eliminated, but the designer must take a very conservative approach and use existing languages as far as possible, and avoid inventing new ones.
  • Do not create a design until you are ready to implement and test the implementation. You should have some overall idea of where you are going, and a overview of the system architecture that will be aimed for, but no detailed design, no detailed description of functional implementation should be written down until the development iteration that will allow that design to be implemented and tests.  The detailed design should cover only as much as is needed to handle the current use case.  The biggest cause of waste in software development is time spend designing things that are not needed or need to be redesigned because of some mistaken assumptions that the design is based on.
  • Software is Plastic. Unlike physical manufacturing, software can be changed in significant ways very easily.   In fact there is plenty of evidence that software is easier to change than the design specifications that describe the software.  Furthermore, software communicates the design more effectively than the specification.  Therefore, you should spend the time to implement the design directly, so that customers can see the details of the design.  If you miss and have the change the design, it is easier to change the software than it would be to change the spec.   But most important, your information about what the customers wants is far better after they have seen the code running.
  • Take the time to code a complete description of the problem in the code that detects and reports exceptional situations. Programmers are often very lazy and throw exceptions with superficial descriptions of what is wrong.  Thinking that they are the only people who will ever see the problem, and they will remember the meaning of the problem from the vague description included.  But in fact more time is wasted in customer support situations because of inaccurate or incomplete error reports than any other cause.   Write every error message is if you are explaining the situation to someone who just walked into the room and has no experience with the code.  The customer, and the customer support team, after all, have no experience with the code.

Courtesy kswenson.

Posted in Others | Leave a comment