HOME  |  NEWS  |  BLOGS  |  MESSAGES  |  FEATURES  |  VIDEOS  |  WEBINARS  |  INDUSTRIES  |  FOCUS ON FUNDAMENTALS
  |  REGISTER  |  LOGIN  |  HELP
Comments
You must login to participate in this chat. Please login.

Good presentation, I appreciate the review, and particularly the anecdotes and discussion ...

An ounce of prevention is worth a pound of cure ...

A stitch in time, saves nine ...

THANKS FOR ARCHIVING THESE PRESENTATIONS!!

Still concerned about the printer display reading ink @ 50% , misleading the casual user ... questioning whether there was a sensor problem, code problem or just what ... sometimes those problems can be aggravating to the novice user (even lawyers, perhaps especially lawyers, get very frustrated at such things). 

I once had a Microsoft Representative relay a story where the help desk received a frantic call from a distraught new user ... looking for the "ANY" key, 'where is the ANY key?? ... (where the instruction was "press any key") ... Hmmm :)

Binary search is a good approach to narrowing in on a problem

(slide 14) "Pick a point about half-way in between ..."

Why did the display read the cartridge level wrong? 

Was the "bad" cartridge re-inserted later to see if a similar failure mechanism and display occured?  ie, was it repeatable

Thank you Gary, very good session!

Preventative measures are good, but they do cut off some learning options in development.

It seems that ENERGY is most important aspect of dealing with debug issues at dev/test environment

Hi, looking forward to explore more of testing techniques

It has taken me awhile to get to this lecture, but better late than never.  We are getting into the testing phase of our design project; so these debugging insights are timely.  Thanks, gary.

Iron

Another good audio lecture

Audio comming in okay

Okay picking up another one

Done with the recording. Gary, thank you for the lecture! Alex, thank you for the logistics!

Iron

While reading the comments for y-day's session, noticed that I'm not the only one late for the session... :)

Iron

Exciting story about problem with a printer... I wonder if someone did it differently - called tech suppport, went through the whole story of "is your computer plugged in"...

Iron

Great that we have these recordings available

Iron

Oops - missed the lecture yesterday due to travel...

Iron

@lansrock dijo, "ya ponte a trabajar moran!". Se aparece que hay algunos que son de habla español. (It appears there are some here that speak Spanish.) Yo puedo hablar español und auch Ich kann nur ein bisschen Deutsch sprechen. (I can speak Spanish and just a little German.) What other foreign languages are represented among the attendees?

Iron

Several of you offered suggestions, alternative ways of debugging my daughter's printer problem. All good. All would have eventually solved the problem.

@danlafleur made a good point distinguishing between troubleshooting systems that used to work (simply find and replace the bad part) versus systems that are being developed and have never worked yet (meaning we still need to change/improve the design so that it will work.)

Iron

@cmeadows6959: Ideally, when an error is detected, the system should either fix it or halt. Don't let it die a slow death. Those are very hard to diagnose. "We hung because there was a bad byte yesterday morning." Error recovery files are good - it's another name/version for data collection that I'm a proponent of.

Iron

@JoeWojcicki: Yes, there is a need and a market for experienced troubleshooters. In my consulting business, I've done that for a few clients. But it's not PLCs that are being replaced. It may be in hw or sw. For one client it was faulty logic in a block in an FPGA that was causing data corruption. For another client, it was suspected that there was a problem in the assembly code. And for another one the hw timing changed causing the fw delay loop to change timing which caused problems.

Iron

@snandu13: I agree, that hw and fw engineers need to be able to work together. Not seeing eye to eye can be good if it is due to a different perspective, which is sometimes needed. But not seeing eye to eye is not good if it means not willing to work with each other.

Iron

@gpapich: The Five Whys would be good to drill down to the root cause but be careful that you don't start drilling in the wrong place. Sometimes a defect is not where we first think it is. The Ishikawa diagram is primarily used in the  manufacturing or service industries once a process is in place. I'm not saying you can't use some of these techniques to debug a design but they are not formally used in that situation.

Iron

@raghu: Is it easier to debug digital systems or analog? For me, I have a definite digital skill set. So digital problems are easier for me. Analog systems would be harder for me but easier for someone skilled in that area.

Iron

@luizcosta: You are correct. It was not a good design to have firmware set then clear the bit, thus creating the opportunity for a race condition. A better design is to have firmware set the bit and then hardware clear it when ready.

Iron

Great presentation.  Thank you Gary and Alex.

Iron

I am a huge fan of explaining your problem to others to reach a solution; I can't tell you how many times I have figured something out while I was explaining it to someone.  It didn't even matter how much they understood about it.  Sometimes, if they have only a cursory knowledge of the subject it actually helps because you have to think more about every aspect of it in it's most basic terms. 

Iron

choose your battles wisely.

 

Dan out.

Iron

And, I hear you about building up the resolve to face superiors (used loosely here) with news they do not want to hear.  It seems like a constant battle that we all have to face.

Iron

You are way ahead of me; I am only 14 minutes in.  Take care, danlafleur.

Iron

OK, archive audio is over and time to earn my keep. See all tomorrow.

Iron

I get the point too. Plan to get the design right the first time instead of using project time from something else because the work needs to be done over again. Courage to say " this needs time to get right".

Iron

As soon as Gary found that switching to color worked, I would've immediately checked the black ink.  I have had this happen to me before.

Iron

Especially if you are relying on repeat business.

Iron

Yeah, you are never going to be able to forsee all possible problems, but I thought that his point was that you have to make some time for troubleshooting, and the thought that you will just always have time to do it better later does not make for a good/successful business model.

Iron

@clia, I got pulled away just before the session started and just now listening to the archive.

Iron

I've learned that project scheduling doesn't make room for expecting unpredictable problems and the time needed to just understand the problem before trying to fix it.

Iron

Things got a little too crazy over here; I had to come back and start the lecture from the beginning now.

Iron

See you all tomorrow.

Iron

Yeah..just like Gary said...people do not have time to make error free but have time to go back and fix...somewhere along that meaning.

Iron

We've been hearing about troubleshooting systems that worked before the failure(s). Debugging is tough because the system may not have ever worked and changing an endless number of parts doesn't improve the situation.

Iron

I came from a third world country where the replacements are hard to find...so troubleshooting skill is a must have...I do understand your point. You can't have a luxury of parts in front line and troubleshooting is a must.

Iron

@THasham, i tend to think that throw away maintenance is a wasteful and shortsighted idea. yes, techs and engineers may get out of the practice of understanding what is broken before changing the part with confidence. I also think what do you do if the replacement isn't available.

Iron

intermittent fauts can be horible. my airforce days of troubleshooting intermittent or recurring problems taught me a lot. look for the obvius things first.

Iron

@danlanfleur...I can't put it better than you mentioned. Nowadays people do not troubleshoot anymore. They just know how to replace, and things are getting cheaper and cheaper everyday. Which make sense on replacing..but our brain get idle though. haha.

Iron

I agree with Ran on Ink cartridge. But I really like how Gary break it down into to process and narrow down the possibilities. We did these all the times and are used to doing these without analysing  the process. or as our third nature.

Iron

troubleshooting seems to be a weak skill these days. Fault isolation is more than the process of replacing parts to make the problem go away. Skill, knowledge, experience and intuition are needed, and being persistent helps.

Iron

Thanks Gary and Alex.

Iron

Gary's experience with simulating the printer engine is similar to me experience in testing Burner Management Systems (BMS). It's dificuilt to have a real operating natural gas industrial burner in the shop, so we built a simulator to act like one to the BMS. All the contyrols outputs and signals in looked like a burner.

Iron

@davep, each session has own slid. You can click on  "View the Full Crriculum Calendar" on top of the page and download next session's slid and reviwe them before class.

Iron

@GStringham – About the ink cartridge problem...  I'd like to offer you another step in your analysis.  The symptom turned out to be a wrong reading of the ink level.  Before undergoing the expense of changing the cartridge, it seems to me that the printer should have been rebooted.  Printers are no more immune to an occasional bit-flip than any other piece of digital electronics.  Neutrinos are flying through space all the time causing such havoc.  Since this is really beside the point of your presentation, there's no need to reply to this one.  Just an observation...

Iron

@davep - click the "Today's Slide Deck" above the post box to the right of Gary's photo and download todays slides

Iron

I need to leave now. I will come back later today to answer the rest of your questions.

Iron

Half-way point is aka "Diviide and Conquer" method of troubleshooting for a problem, or defect.

Iron

@raghu: How do we know failure modes in the beginning? Good question. We can't see into the future. But we can make good guesses based on past experiences. If we think of and prepare for as many possible failure modes as possible, then we will be prepared for the few modes that do occur.

Iron

@apdobaj: Six Sigma is used for manufacturing defects, where their building 1000 widgets and one of them wasn't made right for some reason. That does not apply to design defects. I am not aware of "formalized" debugging methodologies. One needs to know many techniques so that appropriate techniques can be applied at appropriate times.

Iron

are here slides for this talk?

Iron

Whay NOT using rechargable battery? They're same price.

Iron

@s.schmiedl - keep it simple...

Iron

Yet another great presentation

Iron

same with car components, computer components, whatever components. the more you integrate, the more you have to replace at once

Iron

@JM Ashcraft: I didn't replace the ink cartridge first because that is an expensive test if that was not a problem. Once you open an ink cartridge, the clock starts ticking for how long it will last because of trying to keep the ink from drying in the jets. So I only want one cartridge open at a time.

Iron

iFixit (IIRC) tore down the new iPad and gave it the lowest fixability rating they ever gave ... basically replacing a battery is done by replacing the whole device ...

Iron

@tonyh - you you don't know what the h/w can do, you can do anything in s/w

Iron

The whole system of 'defensive design' is to always have access, one way or another, to the key signals in your design.  One of our products was designed by an outside consultant and they way he structured the op amps, we could not see the demod signal.  It causes us no end of grief because we can now only see the signal after the gain and offset stages, not before, so we have limited ability to determine whether the bridge/sensor is working correctly.

 

Platinum

Thanks Gary. It was nice to go through the very systematic approach to  diagnose and resolve an issue.

Iron

Gary, thanks for another great presentation. This really points out the need to understand both the hardware and software in order to resolve issues during the debugging process.

Iron

@gpapich - I will not change my German car to any other car

Iron

@jl: Why did I not print the internal page first? I did look at the printer and saw that there was enough ink (which I found out later was not true.) Probably the main reason I started on the computer/software side was that it was the easiest test to run. "Print it again." Each progressive step required a little more effort so that is probably the main reason I went that way.

Iron

@slk - I don't think that is justification for not providing more pratical maintenance and repairs. :^)

Iron

ya ponte a trabajar moran!

Iron

GREAT LECTURE GARY..  THANKS...

@papich - yes, true, but the German car are going to service less often than the other cars ...

Iron

Thank you Gary! great peresentation.

Iron

what do you think about recovery error files. i mean errors that do no cause a complete shutdown immediately, but can evently make the desige erractic,  slow , or shutdown at some future time.

Good Session. Thanks.

Iron

Thanks Gary, look forward to tomorrow. Thanks Alex.

Iron

I gotta go earlier today, folks. Have fun, and I will take a peek later on onthe chat.\ log.

Iron

Years ago manufacturers were looking for experienced "board level t-shooters"  Is now (XXI Century) are still they are "on the market" when e.g. modules in PLCs are replaced?

Thanks, Gary, great presentation.

 

Platinum

Another good one, Gary...  Thanks!

Iron

Thanks for the presentation. It captures what I've gone through in several system integration projects.

 

Iron

Great presentation and wisdom on testing and diagnostice Gary. Thanks!

Iron

Test Point can be setup on the boardand corresponding with the firmware falg bit for debugging.

 

Iron

Good presentation Gary.

Iron

Slide 22 = KISS principle.  German automotive engineering needs to pay attention here.  Mechanics can't work on vehicles without very specific tools, procedures, excessive complexity.  Lowest-level replaceable parts not very low level...2000 MB S430 Dash display fails? Replace the entire panel assembly ~$1400.00 later.  Happened to my boss.  I drive a Chevy Astro.  Enjoyed gloating at him.  We're friends, we still laugh...

Iron

Thanks for a great session!

Thanks, Gary -- another great session!

Iron

Thank you for today's presentation Gary

Iron

Isn't amazing the impact a great findings, such as the relativity theories, on someones credibility? I wonder how many of Eisntein's sayings we respect as truth!

Iron

hw and sw needs to work to gether to accomplish the design properly.

Iron

With regard to slide 22, I just finished reading "Einstein: Creator and Rebel" by Banesh.  Great read.

Iron

@snadu13 - that is bad

Iron

great presentation Gary!

It is true. Only collobrative troubleshooting can only resolve. But unfortunately firmware and hardware guys dont see eye to eye at least in our company

Iron

When you are working on expensive PC borad, your are not allowd to take risk by changing the part you are not sure is defected.

Iron

slide 22 - should guide your design

Iron

"Defensive design" = Defensive driving on the Intersate HWs in Detroit :o)

Iron

also used the fishbone for a couple of more detailed failure investigations.

Iron

Slide 21: Post-mortems very effective.  We used them extensively in the navy T&E world for test scenario evaluations, any deviations from test plan, etc.  At $100k/day for missile range time, we had to make sure we executed according to plan.  Engineers and the project's value depended on it.

Iron

Gary will come onto the chat to answer your questions directly, after his audio presentation is over. So please start typing in your questions for him.

Blogger

excellent use of the 5 whys.  My use was for component failures on data controllers for helicopters.

Iron

In troubleshooting time, I change a part when I am 70% sure, otherwise don't change it

Iron

Or should I say "solutions discoverd?"

Iron

Yeah @luizcosta.  More problems are solved when falling asleep at night!

Iron

@pshackett: Thanks!  My perspective was as a newbie tech-writer doing a service guide for a piece of robotics liquid handling equipment.  I was fortunate that I was working with a veteran machinist, gearhead and company veteran.  I used the 5-whys very effectively in that case to drive to the essence of why we did certain procedures in troubleshooting the controller to motor connections.

Iron

@syakovac  - this is why you need to work in a team (hw, sw, mech) and define what everybody needs and in the same time can be phisicaly done.

Iron

troubleshooting is in general a potential sleep killer, at least for the geeks. :0)

Iron

I prefer knowing the root cause of a problem I diagnose and solve.  Knowing the cause ensures that the problem is solved and will not return unexpectedly.

Iron

just like, the hardest problem to solve is...when there is no problem.

Iron

Quality black belt engineers use this for finding out the root cause of a problem.

Iron

Test points, software hooks; all good things. But inevitably, after you've placed trap doors wherever you think a problem has potential, there will emerge an unforeseen problem. That's what we're dealing with here.

Iron

@slk -- I have found that if you work with the HW guys, the points that they bring up on the waveform viewers are the ones you really need.  They just don't think to put them in because they have waveform viewers.

Iron

Re: What is "Ishikawa "Fishbone" diagramming"

http://en.wikipedia.org/wiki/Ishikawa_diagram

Iron

@syakovac - if possible, not limit to just "hard to reach" areas. sometimes the most obvious problems are the hardest to diagnose.

Iron

I've used the 5 whys before and the fishbone diagram and it works fairly well with someone that knows what they are doing.

Iron

@syakovac - test point are good to have if you know where to place them

Iron

@gpapich -- What is "Ishikawa "Fishbone" diagramming"

Iron

does the printer start the operation? this is another question cause we can deliver if the problem is mechanical or in the printer or it can be in the way from the computer to the printer

Iron

@jl -- I try to convince people to put observabilty and controllability into those hard to reach places by design all the time.

Iron

A good piece of multimeter comes handy for debugging..like fluke 117 or 87v

Iron

How effective is the "5-Why's" method of isolating the root cause in the electronics world?  Anyone use this or Ishikawa "Fishbone" diagramming to chalk-talk it before replacing parts?

Iron

if you can say it, you can see it..??

@ raghu  - no, both have different problems and approach to solve the problem

Iron

@jsh -- agreed, I was just commenting that I think that was the other person's intent with replacing the cartridge.  I think the difference is 1)  Always find the root cause of the problem regardless of how long it takes or 2)  Fix the problem as fast as possible without the requirement of understanding root cause.  It depends on the personality of the company you work for.  I have worked at both kinds of companies.

Iron

did you try printing a black only doc?

Common sense trouble-shooting, always go for the simple solution first

Iron

problem diagnosis should be part of the design and built into the product. especially data collection type functions that can be turned on in a debug mode in the field.

Iron

It depends what type of digital circuit. If

Iron

In digital systems you can have timing hazards

Iron

Seems everbody has preferred methods. Much depends on your familiarity with the product.

Iron

Common sense trouble-shooting always works, always go for the simple solution first..

Iron

Sometimes, but not always.

Iron

@raghu- Depends on where your experience is centered.

Iron

I am on the "do your best to find the problem"before replacing parts. 

Iron

easier to debug digital systems than the analog ones..right?

Iron

You can seal good cartridge and use later

Iron

I think the linear process worked fine in the example.  My point is that the binary you suggest is not binary if you start at the printer end vs the computer end.  It will still be linear.  Binary you pick the middle.

Iron

The 6 sigma techniques I've found extended the time due for reducing problem issues

Iron

Sounds like the "clear/set" bit issue was related to "racing" and lack of appropriate interrupt scheme.

Iron

If you replace the ink cartidge first, and you find the old cartidge was good, do you throw away a good cartridge?  Wasteful.

Iron

the first shot resolution is made based on the knowledge of the problem space, known symptoms and an understanding of the product

Iron

I am debugging a bad register read problem right now.  The first step is to poke at the design to see if it is the testbench or the design emitting the bad value.  Then, if it is the design, check the read data mux, then the register value.  If you debug it linearly, you are slow at finding the problem (i.e. search backwards from the pin through the spi through the top level, etc.,

Iron

diagnoise and resolve both must use devide and conquer. works every time.

How do we know the failure modes in the beginning?

Iron

Have you ever utilized six sigma or other formalized debugging techniques? If so, what's you take on this seemingly controversial subject?

 

Iron

@syakovac-Starting by replacing the ink cartridge first would not have been very binary either.  Maybe straight to the test pages would have been binary.  Then it is either the computer or the printer.

Iron

it seems it is very intuitive when diagnosing problems to, at first, take a shot at what you might think the problem may be...then follow a more analytical approach.

Iron

we can enter to the test mode of the printer and check

Iron

Before replacing any component, it is better to diognose the fault. Otherwise after a time you changed lots of parts without finding the problem.

Iron

His example seemed a little too much of a linear search for the cause and less of a binary search.  I think that is the point of replacing the cartridge first.

Iron

test post - to see if a cache is holding my text

Iron

I know, good debugging process

Don't lose sight of Gary's ink jet printer story is an analogy to make the point of how to approach failure analysis.  It's not so much about his printer problems.

Iron

ink printers are bad.  cartridges dry out if are anot used.    laser printers are good.

Iron

I would have replaced the ink cartridge first.

This is a common problem with the Inkjet printer.. I own one too

Iron

@Gary - why did in not print the printers internal page first. do you naturally think software fails more than hardware?

Iron

@wthomas46 - Glad you're in the saddle now.  Welcome to the show.

Iron

@wthomas46 - welcome, hope you enjoy

Iron

Sorry I am new to this  program

 

Iron

@wthomas46 - there is no video.  You must download the Slide Deck.  The link is just above your Post box.

Iron

Is not blocked, just refresh your browser.

Iron

Hi All, Sorry I'm late.

Iron

I do not seee any videoo in here it might be blocked

 

Iron

Hello Gary & Alex and everyone..

Iron

Good afternoon, Gary - looking forward to your session today!

Iron

Failure modes...good stuff for mech E and hopefully mechtronics.

Iron

Good afternoon, Alex.

 

Iron

@Gary - break a leg

Iron

Good afternoon all

 

 

Iron

Two minutes until showtime.

Iron

Good afternoon, everybody!

Iron

Hello everybody from good-weather Maryland.

Iron

Ok.. let's talk tech

Iron

hello from sunny miami.

 

 

 

Is [F5] working to refresh audio?

Good afternoon, everyone. Gary, you don't disappoint at all.

Iron

Good morning Gary, Alex and all.

I had to do the Tuesday session from archives and have to say that I missed the realtime chatter.

Iron

Alex,

What kind of info is on the Linkin Group you mentioned, http://linkd.in/yoNGeY?

Iron

Please join our Digi-Key Continuing Education Center LinkedIn Group at http://linkd.in/yoNGeY

Blogger

Be sure to click 'Today's Slide Deck' under Special Educational Materials above right to download the PowerPoint for today's session.

Blogger

The streaming audio player will appear on this web page when the show starts at 2pm eastern today. Note however that some companies block live audio streams. If when the show starts you don't hear any audio, try refreshing your browser.

Blogger

if there's no audio, it's cause your company is blocking live streams. The other thing to try is refreshing your browser but if that doesn't work, you're blocked.

Blogger

Howdy/Hola/Konnichiwa, I hope I don't disappoint.

Iron

Hi/Hey/Aloha, we are waiting for new set of Good Knowledge



Partner Zone
Latest Analysis
Watch as we teardown Amazon's Fire Phone -- the company's first smartphone, designed to compete with iOS, Android, and Windows Phone devices.
Lithium-ion batteries will soon back up the power grid on the Hawaiian island of Kauai, providing the stability to handle intermittent power fluctuations from renewable energy sources.
A relative newcomer to the 3D printing market has developed a 3D printer that can use five different materials in multiple colors for customized creations.
These free camps are designed for children ages 10 to 18. Attendees are introduced to 3D CAD software and shown how 3D printers can make their work a reality. Many classes were nearly 50 percent girls and 50 percent boys.
Take a look through these film and TV robots from 1990 through 1994.
More:Blogs|News
Design News Webinar Series
7/23/2014 11:00 a.m. California / 2:00 p.m. New York
7/17/2014 11:00 a.m. California / 2:00 p.m. New York
6/25/2014 11:00 a.m. California / 2:00 p.m. New York
5/13/2014 10:00 a.m. California / 1:00 p.m. New York / 6:00 p.m. London
Quick Poll
The Continuing Education Center offers engineers an entirely new way to get the education they need to formulate next-generation solutions.
Aug 4 - 8, Introduction to Linux Device Drivers
SEMESTERS: 1  |  2  |  3  |  4  |  5  |  6


Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.
Next Class: August 12 - 14
Sponsored by igus
Learn More   |   Login   |   Archived Classes
Twitter Feed
Design News Twitter Feed
Like Us on Facebook

Sponsored Content

Technology Marketplace

Copyright © 2014 UBM Canon, A UBM company, All rights reserved. Privacy Policy | Terms of Service