@clia (and @THasham): There are several books out there dealing with various aspects of software/system testing. I have not been out looking for what's there so I can't say. I would guess that there is not a book out there about Hackware Testing or similar. It may be because it violates some good practices in software development.
@jrjohns and @caa028: I didn't use the term, "hackware," back then. In fact I didn't realize that what I was doing was that unusual. So in trying to describe it to others, "hackware" was what I came up with because I hacked up the code real quick to run a quick test, then I took it out. So it never became a formal development/test methodology deployed within HP. But yes, "hackware" does have a rogue or underground connotation. I could not find online the term "debug private" used in this context. Another name that describes it is "ad hoc" testing but that is already used and has other meanings within the testing field. Here's some other possibilities: "precision testing," "fleeting testing," "temporary testing," "short-term testing," and "in-code testing." None of those have the same impact that "hackware" does. Any ideas out there?
@caa28: Yes, you can induce some errors at the system level. We could cause a paper jam to occur in the printer but we could not cause the fuser to overheat or the laser to stop working without extensive modification to a pristine system.
@Ran: I also didn't get any awards for having talked to the hardware engineers 18 months earlier which allowed me to get my driver out on time because I didn't have to mess with working around hardware defects.
@snandu13: Humidity is a good way to stress physical items, like the hardware and mechanical components. Most of what I was talking about was stressing the software/firmware aspects. It is rare but I have dealt with cases when humidity affected the firmware, such as changing the timing of mechanical or electrical parts.
@JeromeG12: In most of my career, I used C for the production code language. While everybody thinks that C+ is going to take over, it hasn't yet in the embedded world.
@luiscosta: "Fuzzing," a term I had not heard before. There is a Wikipedia entry for it. http://en.wikipedia.org/wiki/Fuzz_testing. But it looks like it is stuff external to the unit being tested by hammering it from the outside with a bunch of random parameters. That, to me, is very similar to the Stress Testing I was talking about. My hackware is internal to the module being tested and it does a specific task (not random.)
It doesn't make sense to version control hackware. For example, I may, on the fourth loop, pretend that the hw returned 0x3D. I verify it works, then I change it to 0x7D, recompile, and run it again. I may run through a dozen values in 30 minutes to complete my test. Submitting each one of those to version control doesn't seem practical to me. And, if I were to make that a unit test, I'd have to deal with a list of desired values and a loop of some sort. Again, my hackware tests the finer details of my code, details that are so minor that most people probably wouldn't make a unit test for it.
Expanding on the #define DEBUG <level> idea: one can have a global debug variable which is a bitmask, each bit of which controls a particular class of debug information to allow selection at run-time. This global variable can then be set via an initialization file and/or adjusted via a console command, depending on what is available in the system.
Thank you again Gary; some very interesting thoughts and techniques here. I wanted to reiterate THasham's earlier request for book recommendations. I have to run now, but I can check back later here or during tomorrows lecture for your response.
Good points on using #define DEBUG <level> to control how much debug code is in there. I also used a global variable that was accessible from the debugger to control the level. By default, it would be low debug output. But if I wanted to, I could break into the debugger, change that global variable, then have more verbose debug output when running a test. A global variable also allowed hackware to temporarily turn on a higher verbose level when desired.
@luizcosta: Regression testing where there is a list of tests that get run through every time. It is used, for example, every week to qualify a new code drop before releasing it to everybody, to make sure nothing is broken. It is a subset of all the tests because it is mainly a quick test to make sure nothing is generally broken.
@syakovac: You brought up the ATPG scan chain to force a state. And it made me realize that the primary focus of today's lecture was on software. I neglected the hardware side of the equation. Sorry about that. But yes, when testing hardware, my hackware principles still apply. Come up with a quick and dirty scan pattern, run it through and test. What I was referring to with states was software states, where it is easy to write to the state variable to put it in a desired state.
@GStringham – When testing code during the writing of code (hackware), I found it very useful to build into my code compiler directives (at the top of the code) for conditional compiling.That way, I could leave the test code in the final version if I wanted, or I could take it out if I wanted.It really didn't matter because I could set the compiler directive to TRUE or FALSE.
@s.schmiedl: You asked why hackware instead of doing it right the first time by writing a unit test. Good question that seems contrary to what I said. I can test things in finer detail with hackware than I can with unit testing. If I had the time to create a unit-level test for every type of hackware test case I run through, I would create so many unit-level tests that there would not be enough time to run through them all more than once or twice before the product needs to be shipped.
Yes. It is true that DOE will be greater use for improving test effectiveness. But it requires a good amount of statistical knowledge and good training.
@GStringham - You are so correct when you say there are no awards for producing code with zero defects.On more that on occasion I have gone to unit integration with the other programmers on the team and been the only contributor with zero defects, but nobody recognizes the effort/results. No, there are no awards!
I wanted to find out how much attentions validation time should be planned for DOE's (Design Of Experiment).
Testplaning - there are planning factors that need attention on repeatability over multi-runs, VT's and other varriables. Those thoughts will add or reduce validation time, resorces, and other infrustructure. I rely on DOE's to build statistics and modles to help focus on worst case areas. This gives many advantages like varriablility, repeatability, sensativity to help focus on more or focus less on particular areas. If you can not test milions of chips, platforms, and devices you need to plan a statistical target on exceptable margins. You can test ever thing so how do you plan and take some risks.
@s.schmiedl: I don't agree that ALL test code has to be extracted from the product before shipping to customers. But it definitely must not cause a hinderance to the performance of the product. You can tell that Microsoft leaves test and debug code in because when some app crashes, it asks if it can send the data back to MS for analysis. This really helps them understand what is going on out there.
@GStringham - I can't speak for anyone else here in our forum, but I found today's session with you the best session for me since we started back in January.Thanks Gary for such a good session.
@clia: Yes, some companys do throw out a little tested product and let customers test it. It's not a business model I care for but it does work for some.
GARY: If you use "HACKWARE" isn't better to incorporate that in the versioning process where the "librarian" of documents keeps in track of all inserted "bugs" used during the test phases? Another point is "fuzzing" is already used instead of "hackwaring" and there are books and research done on it..
Would it not be better if we use start-up diagnostics and other built -in diagnostics of all the critical functions ? Unfortunately the diagnostics is never complete in a system
Testplaning - there planning factors that need attention on repeatability over multi-runs, VT's and other varriables. Those thought add or reduce validation time, resorces, and other infrustructure. I rely on DOE's to build statistics to help focus on worst case areas. This gives many advantages like varriablility, repeatability, sensativity to help focus on more or focus less on particular areas.
It's too bad that the name Hackware was chosen. My management chain would have a fit if I suggested taking a Hackware approach (without knowing what Hackware was.) Perhaps a more subtle name? :-)
You say you took the hackware out - why not control the test code using compiler directives or #defines and utilize a "test" branch in your code repository?
For manufacturing testing of digital logic, the part is configured as a shift register so that all registers are accessible, controllable, and observable in all states
how do you test the code later again, if you take out the hackware? why not do it right and write a (unit) test for every code path as you go along and create them?
@jmh- With the option to stream out data from a serial port, it sounds like your company also thinks the product should be tested before shipped, unless you do that just for the customer.
@ jmh - you don't want to have just your customer feedback. they are screaning when they have a problem, not when everything is ok. you need to do your 'homework" first in house.
@jmh- I agree; they are a tremendous resource! But, my point is that they should not be the only resource. I cannot see this as anything other than short sighted.
@Greekeng- But, where is that line drawn between risking losing clientele for missing the almighty deadline, vs. risking losing clientele for putting out sub-par products?
Yes then after collecting all this data, I had to "de-engineer" the data so management could understand, then they'd cut my time, pulling my hair out.....
We have option to stream data out a serial port in our products for testing and performance purposes. We then organize it, plot it, save it using excel.
The streaming audio player will appear on this web page when the show starts at 2pm eastern today. Note however that some companies block live audio streams. If when the show starts you don't hear any audio, try refreshing your browser.
By refining topologies and using new fluid technology, Moog's new peak sine drive controller increases available power without increasing controller volume.
Lantronix Inc. has expanded its line of controllers for sensor networks with the release of a rugged controller that improves management of automation systems used in a number of industries, including manufacturing, oil and gas, and chemicals.
Inspired by the hooks a parasitic worm uses to penetrate its host's intestines, the Karp Lab has invented a flexible adhesive patch covered with microneedles that adheres well to wet, soft tissues, but doesn't cause damage when removed.
From Dell / Intel® New Paradigms in Design Work Scott Hamilton, vertical market strategist for Dell Precision workstations, 5/2/2013 3
Early in my career, I worked as a draftsman and remember the days of drawing on vellum with numbered pencils and Mylar with plastic lead. This was a fun experience in the sense that I ...
I've been using workstations for more than 10 years and love finding ways to get more performance from my system. With demanding professional applications that require more power each ...
A lasting memory from my first job as an engineer in an auto assembly plant is standing on hard concrete at six in the morning, vending-machine coffee clutched in hand, listening to ...
A quick look into the merger of two powerhouse 3D printing OEMs and the new leader in rapid prototyping solutions, Stratasys. The industrial revolution is now led by 3D printing and engineers are given the opportunity to fully maximize their design capabilities, reduce their time-to-market and functionally test prototypes cheaper, faster and easier. Bruce Bradshaw, Director of Marketing in North America, will explore the large product offering and variety of materials that will help CAD designers articulate their product design with actual, physical prototypes. This broadcast will dive deep into technical information including application specific stories from real world customers and their experiences with 3D printing. 3D Printing is
To save this item to your list of favorite Design News content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
Tweet This
7 
