WingNews logo WingNews GitHub [2]

jayjohnson's comments

jayjohnson | 7 years ago | on: Riot Games Approach to Anti-Cheat

Good timing, I am using my own AI (keras + tensorflow) stack to predict in-game hackers on ARK Survival Evolved with an AWS EC2 instance. Here's some background on the fully open-sourced stack: https://github.com/jay-johnson/train-ai-with-django-swagger-... with docs http://antinex.readthedocs.io/ I would love some players, but I'm still load testing how many players the game server can use + make real time predictions without impacting the game. Reach out if you want to try it out!

jayjohnson | 10 years ago | on: A Docker Container Pattern – Compose Configuration

These are great points, but with the post I wanted to share how I approach designing and building docker containers, because I hope it saves others time building them. When I started trying to deploy containers on production without Docker Compose it was painful. Now that I know what that time sink feels like, I try to focus on using configuration management during initialization inside the container instead of at the Dockerfile RUN directive (which requires a rebuild).

Back to your points, can I ask if you are interested in using this container approach to build a database container cluster in docker and then benchmark against a working imgadm(1M) + SmartOS environment? From the docker-side when I scale out a cluster of database runtime-only containers, I utilize the volume mounting attribute that is commented out in the repo's compose file (https://github.com/jay-johnson/docker-schema-prototyping-wit...) and then host the database files outside of the containers in a persistent storage location that is mounted + available on the hosts where the docker containers are running. As for building the database container for this proposed solution I would still utilize this Docker Compose-centric approach in development and then in production because I can build the container one time and not deal with a rebuild which invalidates testing a container within my DevOps artifact pipeline.

Wrapping up, I live more on the application development side so I cannot speak specifically to SmartOS or imgadm(1M), but I know the docker documentation and community are pretty helpful for getting/finding technical solutions that helped me build + launch products (go team!). If you are interested we can discuss more details about how I would approach bench marking these two database environments. I am a big proponent of testing everything and I would enjoy discussing how to tackle a db perf/load/ha + test harness like my message simulator (https://github.com/GetLevvel/message-simulator).

Feel free to connect with me on LinkedIn anytime: https://www.linkedin.com/in/jay-johnson-27a68b8a

jayjohnson | 10 years ago | on: A Docker Container Pattern – Compose Configuration

Agreed. That's a great point when it is a simple use case, and how I started. Now that I build for Docker Compose out of the gate, it allows me to deploy a composition across a multi-host Docker Swarm (usually just changing the network driver to overlay + utilize labels) with 1 command, and more importantly I do not have to rebuild my container on production. Back in docker 1.9, I found myself running too many variations to get the same behavior that Compose natively handles with the new labels + overlay functionality (reference http://blog.levvel.io/blog-post/running-distributed-docker-s...). I was using `docker run` to deploy across specific hosts using manually-assigned env vars invoked with a docker run:

# docker run -itd --name=AppDeployedToNode1 --env="constraint:node==swarm1.internallevvel.com" busybox

# docker run -itd --name=AppDeployedToNode2 --env="constraint:node==swarm2.internallevvel.com" busybox

# docker run -itd --name=AppDeployedToNode3 --env="constraint:node==swarm3.internallevvel.com" busybox

RabbitMQ Example - https://gist.github.com/jay-johnson/2673ce4df42317667908#fil...

Looking back I feel like it was a lot of effort to deploy 3 busybox instances or that initial RabbitMQ cluster…and that effort to handle the “production deployment case” as early as possible is what set me on the path to the new Docker Compose-centric approach discussed in the link above.

jayjohnson | 10 years ago | on: A Docker Container Pattern – Compose Configuration

I am pretty new kubernetes. What are some of the gaps with docker compose vs kube's deployment orchestration? I am pretty happy with docker 1.10.3, but am always interested in hearing about something better/cleaner (I don't know what I don't know). I'm looking over your github for some samples at the moment.

jayjohnson | 10 years ago | on: A Docker Container Pattern – Compose Configuration

I totally agree and this is not a great idea for new comers. I do find it easier to debug things with all the bells and whistles turned on, but yea the CAP adds are NOT production ready. I will remove them to prevent confusion for those looking to get started with docker.

jayjohnson | 10 years ago | on: On asking job candidates to code

Nice. I had the same experience with DO last September...The same month Braintree asked me to write a ruby project that took about 3 days and then on the call only asked about python sqlalchemy database queries. I don't know what the disconnect is with these larger companies and their HR staff? By the time that third one asked me to write a node.js rest api from scratch. I just chuckled and said thanks for your time...I doubt I will commit more than a few hours in the future to any throwaway project for just a call back. Why can't we get paid for the time spent writing sample code like a freelancer...If it isn't good enough pay for my time (up to a max) and both parties can move on. Time is more valuable than the risk of getting passed that initial callback/more screens.

jayjohnson | 12 years ago | on: Ask HN: Idea Sunday

How intrusive would the developer environment have to be? We're a "host your own REST service" development shop so deploying and testing your own GitHub repo is done with a dashboard from a web page. In our eyes, everything outside of code development and code hosting should take place in a web page complemented with command line tools for advanced control.

The feedback we have heard is the editor of choice is near-and-dear to a developer and would be difficult to force a new tool on anyone to edit code. Code hosting has a good solution with GitHub too. Which leaves testing and deployment up to the platform. Since we focus on giving two free REST services for publishing your own Jobs we put the deployment and testing into our web dashboard (and cli tool). Testing a REST API is easy and already integrated into our dashboard similar to Postman. The only code a developer has to write is a Job which is just 3 simple parts (inputs, outputs, and tasks). For us we wanted the .Net debugging power for every developer from our dashboard so we built it to identify tests that fail validation based off your test inputs and outputs versus your test's expected outputs.

Let me know what you think here's our beta vision based off our feedback for quickly developing and deploying REST API services for your own Jobs: https://flowstacks.com/developer/

Cheers

jayjohnson | 12 years ago | on: Ask HN: Idea Sunday

Interesting idea. So we're in Beta and looking for good ideas like this to build showing our technology. For background, we take a user's GitHub repo and deploy them into REST API endpoints hosted on EC2. Each REST API is already paired with a database with SQLAlchemy. Clients invoke their REST API to publish their code as a "Job" requiring 3 parts: inputs, outputs, and tasks to perform (states). Those tasks can be whatever is necessary for your tdd and we test Jobs directly from our Web Developer Dashboard or with our cli.

I could see something like a Final Exam repo setup where your students fork it for development and then testing based off the same code and tests are shared already in the repo and can be invoked for grading from our cli or the Web Developer Dashboard.

Some More Information on FlowStacks (https://flowstacks.com/)

FlowStacks Hello World Example https://github.com/FlowStacks/hello-world

FlowStacks Tests https://github.com/FlowStacks/hello-world/tree/master/tests

Endpoint-specific Database Schema https://github.com/FlowStacks/hello-world/blob/master/schema...

Example Job Inputs https://github.com/FlowStacks/hello-world/blob/master/src/ra...

Outputs: https://github.com/FlowStacks/hello-world/blob/master/src/ra...

Example Job Tasks to Perform (States) https://github.com/FlowStacks/hello-world/blob/master/src/ra...

We also support Jobs integration with Redis https://github.com/FlowStacks/hello-world/blob/master/src/ra...

Contact me if you have more questions.

Cheers,

Jay

jayjohnson | 12 years ago | on: Our Startup can help other Startups build an MVP

All great points, and just goes to show we need to be updating the website and docs more actively. So for starters here’s where we are: we're in week 7 of beta where the core focus is giving developers a free public sandbox for two hosted Endpoints + a single database per deployment all for free. We have two tools for using FlowStacks:

1) The Developer Dashboard (https://flowstacks.com/developer/) is for testing new code, managing all endpoints, deployments, database schema (coming this week), and account details from a browser.

2) The command line interface (https://github.com/FlowStacks/cli/blob/master/fs.py) for full API capabilities, scripting, and the latest error handling. We are a command line shop, so we use this to find bugs, and it already reports things like bad JSON syntax, API Validation, Python syntax, Exceptions, and as many other silly errors that really annoyed us.

We’re still building the advanced deployment and database management into the website, but all of the tools are already working from the command line. We are also close to releasing a working Django webserver that uses any Endpoint-level API file (https://github.com/FlowStacks/hello-world/blob/master/api/wo...) for quickly mapping Javascript strings through Django into a working Job[1] hosted on any FlowStacks REST API. This factory style approach to building Jobs extends beyond Django, but since we’re a Python shop we went with it first. We planned on knocking out Rails next, but we are flexible.

Why FlowStacks? As developers, we wanted to have a Job hosted on a REST API where the API Inputs/Outputs can be Job-specific, and we did not care where the Job executed. For us, this API standardization has made us able to quickly build our own dashboard using and testing FlowStacks Jobs like any developer. We want the Dashboard arming developers just like .Net for debugging where streamlined testing drives features out faster, and it has to be simple to use (no Jenkins/Travis it’s all JSON black box testing similar to Postman). We built the command line interface for handling build regressions and showing how to use the FlowStacks User API and REST APIs from the command line. We also built the cli and dashboard for deploying your own GitHub repository into a REST API where your Job code defines the hosted API(dashboard build is coming this week). Jobs are easy to write. There’s only three parts to a Job: inputs[2], outputs[3], and tasks to perform (states)[4]. This allowed us to build some of the first Jobs like our Twitter Poster Job[5] because we did not want to login to Twitter. We also include Redis integration for any Jobs[6] out of the box already, and we use redis extensively for caching and publisher/subscribe patterns.

Our data policy is very open based off feedback we hear. Right now, we use Python’s ORM SQLAlchemy for any schema integration into your REST API’s database instance (https://github.com/FlowStacks/hello-world/blob/master/schema...) and Jobs can also use your new/legacy database(s) outside of FlowStacks. We just need a network connection. Here’s our example included in the Hello World for setting up a REST API Job to establish a connection to any database outside of FlowStacks: https://github.com/FlowStacks/hello-world/blob/master/src/ra...

Post Beta we plan on launching the on premise offering for Enterprise solution development and deployments on site. We can support these if needed, but for now we have it as a lower priority for us just trying to build a larger developer audience. Lastly, we are looking to build something with this, and we always want to hear some more great feedback like above.

Email me and let’s set a time to discuss this.

Cheers, Jay

More References:

1 – API Job fields: https://github.com/FlowStacks/hello-world/blob/master/api/wo...

https://github.com/FlowStacks/hello-world/blob/master/api/wo...

2 – Example Job Inputs: https://github.com/FlowStacks/hello-world/blob/master/src/ra...

3 – Outputs: https://github.com/FlowStacks/hello-world/blob/master/src/ra...

4 – Example Job Tasks to Perform (States) https://github.com/FlowStacks/hello-world/blob/master/src/ra...

5 – Sample Twitter Job: https://github.com/FlowStacks/social-workflow/blob/master/sr...

6 – Redis Job Integration https://github.com/FlowStacks/hello-world/blob/master/src/ra...

7 – Postman https://news.ycombinator.com/item?id=7615365

page 1