There's something SERIOUSLY wrong with Charlie!

There’s something seriously wrong with Charlie.

When I fired him up today, I was confronted with the two eyes and the antenna light running at greater than maximum brightness and the board totally unresponsive except for the power button and servo’s.

I am currently exploring the possibility that the firmware has gone all pear shaped and I am trying to remember how to re-flash it.  I distinctly remember a feature on the desktop that would allow the operator to re-flash the robot’s firmware, and I’m still looking for it.

I tried the “Classic” Raspbian For Robots, and now I’m re-downloading Dexter OS.

Right now Charlie’s controller board has been removed and attached to a Raspberry Pi 3 so I can try to re-flash the firmware.

The only other possibility is a serious hardware fault which might be repairable, but I really don’t want to go there if I don’t have to.

:pray:

2 Likes

Update:

It appears that the controller board is still functional, but the control of all the neopixels has been lost.

It’s time for the schematic.

2 Likes

Good news! Hope you can fix Charlie up like new.

Perhaps a sunspot or Crudstrike.

1 Like

Possibly what the auto mechanics call G.F.S. - Green Fuzzy [Stuff].

I cleaned up some flux residue and removed some excess superglue, and it works - sometimes - and it fails sometimes as well.

I have it narrowed down to a small area near R14, the 220Ω isolation resistor for the string of Neopixels.  I’m suspecting either a cracked solder joint or a cracked trace.  My problem is that I can’t get high enough magnification to see for sure what’s actually going on.

2 Likes

Found that the 220Ω resistor is damaged.  Possibly by my efforts to rework the area.
 

So I will have to go get one.  The little beastie isn’t any larger than 1mm x 0.5mm; barely large enough to hold in my tweezers!  I’ll probably get four, just in case. . .
 


 
If you look at the small square pad on the left, there appears to be an indentation in the trace leading away from it.

That might have been the original defect.

2 Likes

Your SMD removal / pad cleaning techniques and tools?

2 Likes

A small tip on my soldering iron, a pair of tweezers, solder-wick, and luck.

The rest of the damage was caused by my attempts to remove excess super glue, (the white residue), when I glued the big capacitor out of the way of the Pi-4’s fan.

1 Like

I was reading about folks using some sort of heat gun and low temp solder, plus the solder-wick. I blew the “I channel” of my SDR receiver yesterday and folks talk about needing the heat gun approach due to pads too close for using a soldering iron.

2 Likes

Oh no - best of luck with getting him fixed.
/K

2 Likes

And every word of that is absolutely true.

The lowest temp solder I have found is pretty much standard 60/40 solder.

Alloy_Temperature_Chart.pdf.txt (133.6 KB)

This is a chart of the melting point of various solders.

=============== Update ===============

Charlie’s controller board appears to be working again.
 


 


 

Ultimately I had to drill two 0.5mm holes into the PC board to isolate the LED signal trace from the rest of the PCB.  Then I connected a 2.2k resistor to pin 49 on the controller IC, ran a jumper from the resistor through the two holes and thence to the power Neopixel on the top side of the board.

The biggest problem is seeing the components while trying to solder them with a micro-tip soldering iron.  (Pro tip: If you’re going to be doing this somewhat regularly, invest in a soldering microscope and a heat gun.  The absence of these tools significantly reduces the quality and accuracy of my rework.)

Another problem is confusing 450° F, with 450° C.  (Pro tip: 450° C is WAY TOO HOT for PCB rework causing me to lift lands all over the place.)

Ultimately I got things working again and tacked everything down with hot-melt glue.

Here’s Charlie’s controller board working correctly again.
 


 

Next Steps:

  1. Reassemble and test Charlie.

  2. Disassemble Charline and find out why one of her front blinkers doesn’t work.

2 Likes

Congratulations!

That you could diagnose the issue was impressive. That you have fixed the issue is immense.

I can see Charlie is “beaming” with joy.

2 Likes

It actually wasn’t that difficult.  Once I had proven it wasn’t a firmware corruption issue, the troubleshooting was relatively straightforward.

I’ve never claimed to have the software chops that you or Keith have.  (Especially you.)  Staring at your code usually gives me a headache. . . (But it’s extremely educational!)

However, one thing I will always bank on and back up to the hilt is my hardware skills.  I won’t say I’ve never been beat, (and yes, I am generally successful), but even when I do get beaten down, the other side crawls away with a serious hurtin’ too.

2 Likes

Preliminary tests using GPG O/S indicates that Charlie is working properly.  I haven’t tested the sensor inputs or i2c, but I’m confident that they will work.

P.S.
@cleoqc
What I REALLY want to see is a component layout diagram.  Since none of the parts are labeled with component designations, it’s difficult to find out which of the 115 different components with the same value is the specific component I am interested in.
(:man_facepalming:)

A PCB layout would be the sprinkles on top of the cherry on top of the icing on top of the cake!  (And it would make repairs MUCH easier!)

2 Likes

Agree.

That’s fantastic! Great work.
/K

2 Likes

CC: @cleoqc

Update with Charline:

The problem with the left blinker was (yet another) soldering defect.  (Pay particular attention to the right-hand side of the LED.)
 
IMG-20240727-WA0004
 
IMG-20240727-WA0005
 

You will note that the solder on one side of the LED has not flowed properly and, in fact, the LED isn’t even flat to the PCB.

This isn’t the first assembly workmanship defect I’ve found with Charline.  For example, look at the quality of the soldering on the through-hole connector pins.
 


 

 
(The one sloppy joint where the black mark is where I connected to the Pi_Running signal.)

Here’s the repair:
 
IMG-20240727-WA0008
 

I ended up re-soldering most of the through-hole components and some of the surface mount devices.  The bald statement “I’m quite disappointed with the build quality” is a masterpiece of understatement.

Not only do we have to deal with whatever problems people come here with, but now we have to be concerned with the potential issue of defective devices due to crappy quality control.

Now, don’t get me wrong, I don’t expect MIL-883 or DOD-2000(+) levels of quality; that’s not reasonable for a relatively low-cost commercial device.  However I do believe we have a right to expect reasonably workmanlike construction that doesn’t have things falling off the PCB or solder smeared all over everywhere.  This I place directly at Modular Robotics’ feet as they are ultimately responsible for the quality of their products, no matter where manufactured.

Maybe some of their products were “Ghost-Shift” products?

Viz.:
https://www.bunniestudios.com/blog/2010/microsd-card-failures/

https://www.messynessychic.com/2021/07/08/the-secret-world-of-ghost-shift-midnight-manufacturing/

What say ye?

1 Like

Chumby - I remember those - I only managed to acquire a “virtual chumby” but it was definitely a trail blazer.

Interesting to hear an inside story from the maker.

2 Likes

A quote from the Ghost Shift article

Blockquote As Alibaba co-founder Jack Ma told the BBC, “The problem is that the fake products today, they often make even better quality, better prices than the real names. The exact factories, the exact same materials, but they do not use their names.”

I didn’t realize ghost shifts were a thing; in hindsight I guess they were inevitable. And if the fakes are better quality than the originals, that’s really primarily a problem for the brands selling overpriced merchandise. Of course for customers it’s also a problem since it can be hard to tell the quality items from the inferior ones (‘fake’ or real)

/K

2 Likes

I’ve run into this in the past when ordering replacement batteries for my cell phones.

I ordered them from Amazon and ended up having to order and return two of them, (one an obvious fake, the other a more subtle fake, until I finally received what appeared to be a genuine battery, (or maybe a very clever fake), that worked properly.

I’ve also received fake automotive suspension parts from them too which I returned.

It’s a pity that you can’t even trust someone like Amazon nowadays.

Truth!

Hopefully this will help moderate the prices on the name-brand products.  I don’t think so because there is a certain “panache” to having known expensive brands, silly or not.

1 Like

I’m sure the 6.5M non-iDiot computers crippled for days recently are loving all that money they saved.

2 Likes

And all the non iDiot computers that run Linux or don’t bother with those fancy updates.

That’s why I (usually) try not to be a “first adopter” on things, especially updates if at all possible.

Windows just happens to be a “target rich” environment.

In this case, it had nothing to do with being a “Windows” system, I think it crashed Linux systems using that service too.  All because someone never tested a “null” filled update package.

The problem is that this subscription service operated as a kernel process in ring zero.  If something like this happened on a Mac, it would crash too.

That particular service has just seen its credibility totally tank.

2 Likes