datahead's comments

What is the general outline of going from a model of a craft (drone, sailboat, etc) to building a sim that can do reinforcement learning over a physical object interaction with its env?

I want to start playing with models, sims and collected data for sailboat racing- I know the RL/data science stuff, and I assume a good model of your craft takes time to build, and can be improved with collected data. What are some areas to explore when chaining model -> sim -> RL for performance?

I realize this is an extremely complex topic, with several PhDs worth of potential input- if you had to explain to someone technical what it looks like and where to keep digging, what would you say?

oh, your luser account is 'letsdothisagain', great thannnnks. I'll take care of that right now.

Curious what recent work you are referencing? Lots of new so I don’t want to miss things. In my experimentation I basically wrote off LLMs from doing any math directly. That includes aggregations or precise calculations. I’ve been playing with sql generation and allowing the db to do its thing.

Hi holoway, I've been following your project for a couple of months now. How do you compare yourselves to upbound/crossplane?

IME crossplane has been a "much better terraform" and also borrows from some of tf's open source provider code. Seems to be one of the best IaC pattern for k8s centric shops.

Pros: - adoption of k8s core engine, state mgmt

- model everything as a CRD, consistent definition pattern both infra and app

- open source

- great UI when layered with argoCD

- declarative

Cons:

- steep abstraction learning curve (for me anyway)

- docs lacked key context for newbs (also getting way better, big efforts here)

edit: formatting

How did you approach testing this? Anything to watch out for when defining abstractions? I'm following Crossplane's abstraction model.

@cyberax said, "My problem with the current IAS systems is state storage. It should not be needed! Instead, the IAS tool should introspect the systems it's managing and build the necessary state on the fly.

@firesteelrain said, "you can do that through abstraction. You "include" your Terraform Azure Provider or Terraform AWS Provider. At the end of the day, your module needs to know what it’s interacting with but not the higher level of abstraction. We have done it at my work to make it cloud agnostic just in case we need to go to another CSP"

Single ops eng in a 3 person startup here. Ops eng is only one of my hats right now :) I found crossplane to be a solid tool for managing cloud inf. My assertion is that "the only multi-cloud is k8s" and crossplane's solution is "everything is a CRD". They have an extensive abstraction hierarchy over the base providers (GCP, TF, Azure, AWS, etc) so it's feasible to do what firesteelrain did. My client requirements span from- you must deploy into our tenant (could be any provider) to host this for us.

I can setup my particular pile of yaml and say - "deploy a k8s cluster, loadbalancers, ingress, deployments, service accounts (both provider and k8s), managed certs, backend configs, workload identity mgmt, IAP" in one shot. I use kustomize to stitch any new, isolated environment together. So far, it's been a help to have a single API style (k8s, yaml) to interact with and declaratively define everything. ArgoCD manages my deployments and provides great visibility to active yaml state and event logs.

I have not fully tested this across providers yet, but that's what crossplane promises with composite resource definitions, claims and compositions. I'm curious if any other crossplane users have feedback on what to expect when I go to abstract the next cloud provider.

cyberax's note on state management is what led me away from TF. You still have to manage state somewhere, and crossplane's idea was- k8s is already really good at knowing what exists and what should exist. Let k8s do it. I thought that was clever enough to go with it and I haven't been dissapointed so far.

The model extends the k8s ecosystem, and allows you to keep going even into things like db schema mgmt. Check out Atlas k8s operator for schema migrations- testing that next...

I also like that I can start very simple, everything about my app defined in one repo- then as systems scale I can easily pull out things like "networking" or "data pipeline" and have them operating in their own deployment repo. Everything has a common pattern for IAC. Witchcraft.

Back in my early days of data eng for ML pipelines, I stumbled onto Drake- and it opened my eyes to managing pipelines as DAGs. This pattern is supremely effective, and I try to teach anyone who might benefit.

https://www.factual.com/blog/introducing-drake-a-kind-of-mak...

Don't overlook the comment about form. Form is everything, weight is secondary. Get a professional trainer to help you setup your form once you move to barbell work and then you can go from there on your own.

r/homegym will have some good recs for gear to get started. Plenty of bodyweight training you can do too- great for turning on your main muscle groups (youtube).

A barbell and some plates w/ rack will cover the main lifts. Squats, deadlifts, presses (chest and shoulder) are the first ones to build. I recommend Starting Strength for an on-ramp to get going. Adjustable dumbbells are great too- plenty of options out there. Some of this equipment is expensive, look for used and consider it an investment in health.

Happy Lifting!

I did as well, and was rooting for him.

One of my favorite lessons "learned the hard way" by someone else. You fundamentally cannot run a business on poor data practices. It will sink you before you realize what happened.

https://www.canadianbusiness.com/the-last-days-of-target-can...

I'm curious if our interactions could be biased. For example, we're clients of RH (RHEL & Openshift) so I'm guessing we interact with more presentable humans. Everyone I've met has been genuinely wonderful and supportive across eng, sales, product, TAM, etc. One of the few vendors I enjoy working with.

I used inclusive in this context to mean that a junior engineer is expected to speak up and offer input in an executive meeting if they have something to offer. Cutting through hierarchy, "good ideas come from everywhere". This could be idealistic but it helped me in eliciting input.

I don't know what it's like to be on the inside or close periphery to daily core operations at RH. I agree and understand completely what you are saying. "Free speech" has (recently?) been a guise to act terribly toward each other in a number of settings. Trolls gonna troll. It's important to maintain decorum in the workplace and a culture that defies that can turn toxic quickly.

I like Whitehurst and have had the chance to meet him a few times. If you want to understand his ethos check out "The Open Organization". It details his transition from Delta to Red Hat and the transformation of his leadership style being around a lot of smart engineers and team players ready to constructively challenge each other. Good read for anyone interested in building inclusive company culture in tech.

Honestly, I thought IBM would be feel less rudderless with him at the helm. I wanted IBM to become more 'red' and not RH to become more 'big blue'- but it appears this isn't the case after all... seems like a short tenure.

Anyone know more about the new leadership?

I support getting the sampling rate and quality up. To what level? Idk.

Have you ever been in a room with a DJ playing low quality music on a decent sound system? It's awful, grating. Now at least your everyday aspiring DJ has access to higher bitrate tunes and doesn't need to blast low quality junk out of the PA system.

Overnight on the USS Yorktown (CV-10) was awesome, one of the highlights of all the scouting trips we took. It felt like we had full run of the ship, and my buddies and I explored quite a bit. My first impressions of WWII history and Battle of Midway where USS Yorktown (CV-5) sank were from the tours and videos shown that day.

First, if you haven't worked in a large "non-tech" enterprise, you might be surprised to find out it's just 100 startups with a common name and financing. The bigger the company, the less technical cohesion across teams. It's a blessing and a curse. I'm technically in a "business unit" as opposed to IT, so we're more focused on tangible KPIs. When you turn around innovative results in the biz domain consistently, no one really questions your tooling.

We started a new space Data Eng/ Data Sci and then it became critical. We've been able to maintain products by building long term platforms/systems that meet current and (anticipated) future needs in clojure. Initially, product and platform/system was all the same team. We've grown a little and now individuals tend to focus more on one or the other, but we're still <15 eng and shipping daily.

We maintain everything we've built to date, and we planned for that going in. Clojure's other superpower is that it is stable AF over long periods of time, handles refactors and testing well.

As for signoff in adopting clojure... our VP is old hat in clojure/lisp. We got lucky.

Great article. Even in a large enterprise, where the prevailing organizational ideology centers around building the largest headcount you can for political sway...

I've kept my team small on purpose, and we were able to be very effective by adopting clojure. We have sway because we deliver- and that's a different type of leverage than headcount. We don't get thrown every hot potato, instead we're consistently aligned with the critical portfolios.

A small dedicated team with tools such as clojure will outdeliver, and outcompete a larger team who cannot remain agile and require significant overhead to manage.

I'm basically rehashing PG's 'Beating the Averages', but it's been my experience as a dev mgr. http://www.paulgraham.com/avg.html

I found this [1] breakdown helpful to understand the legal position of this hypothetical.

> unlimited [...] discussion, planning and coordination

turns into evidence once "intent" and an "overt act" thresholds are crossed, for all involved.

[1] https://criminal.findlaw.com/criminal-charges/conspiracy.htm...

Interesting presentation that explores your question. +1 intersection from this commenter.

Tim Ewald - Clojure: Programming with Hand Tools https://www.youtube.com/watch?v=ShEez0JkOFw

Hacking Healthcare (HC specific) and The DevOps Handbook (general) are the books I buy for everyone joining the team.

I recommend Eric Topol's books as well, including The Creative Destruction of Medicine and Deep Medicine. Lots of food for thought.