top | item 27116079

(no title)

amboar | 4 years ago

> It was eating characters, which is the second worst thing it could do.

Okay, that's interesting. Did you file a bug with your system's firmware provider or with upstream?

You can file it against obmc-console at https://github.com/openbmc/obmc-console/issues

> Digging into the source, I found a really wonky software buffer... which kind of blew my mind - since hardware buffers and flow control exist.

Yep, it's how OpenBMC provides the console via IPMI, Redfish, SSH and on the commandline, as well as routing the console out to the connector on the rear of the chassis.

> I don't remember if that was a case where python was somehow in the pipeline, but that was unfortunately very common in OpenBMC.

Generally there hasn't been any python in the console handling pipeline. That said, python, while especially slow on a BMC, was pretty important for getting the project off the ground.

At this point OpenBMC has been python-free for several years.

> I also had a lot of trouble getting the thing to honor the most basic settings, like baud rate

There's some nuance to this, as depending on your system design the console may be coming from the host to the BMC via Aspeed's Virtual UARTs (VUARTs). The way the VUARTs work is the BMC and host are connected to either end of the two FIFOs between the UARTs' APB and LPC/eSPI interfaces. As such there's no baud rate as no data is clocked out in RS-232 fashion - the data is transferred as quickly as either side can access their register interfaces.

This has caused issues in the past with control flow that lead to data integrity issues:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

> But it is harder to confidently place the blame on that one. Is it OpenBMC's fault that the software provides no clue as to where things are going wrong in the serial plumbing? Is it the AST2500's fault that designers are routing every serial line through it and making it impossible to eliminate as the cause while troubleshooting?

I agree it can be difficult to debug where things are going wrong in the console pipeline.

In concept a serial console should be simple, but when you bring in requirements like various kinds of SoL, it starts to get more complex.

discuss

order

sennight|4 years ago

> Did you file a bug with your system's firmware provider or with upstream?

Nope, I patched my code to mainline the bits directly to where they were going.

> ...it's how OpenBMC provides the console via IPMI, Redfish, SSH...

Which is why I didn't bother with upstreaming :) I'm all for it generally, but in this specific case I saved myself and others time (distilling a reproducible test case, crafting code for public consumption, arguing my case, etc); I'm not the target market - I'd cut the BMC traces if I could without wrecking the machine.

> ...was pretty important for getting the project off the ground.

Sure, but far too close to the metal for my liking. But then I've always bristled at the sight of it being used to replace far more constrained tools - where constraint is desirable (like build systems).

> There's some nuance to this...

That was mercifully high-level, but touches on an major concern of mine that is more to do with Aspeed than OpenBMC: backdoored blobs. The AST2500 is constantly sniffing a software defined uart for a magic sequence to drop it into the debug shell... this is the opposite of what I'm looking for in a secure or performant system.

> I agree it can be difficult to debug where things are going wrong in the console pipeline.

Yup, without even considering the client hardware/OS/application stack, or the getty process on the host you actually want to talk to. I've actually been researching a bug that made it into BSD4, the fountain head. Nobody seems to have noticed that a bunch of terminal attributes, when set from gettytab, stopped having any effect 30 years ago (aside from a single potentially related unanswered complaint on Usenet in 91).

http://students.engr.scu.edu/~sschaeck/misc/aspeed-edac.html