> Rather than simply transliterating source COBOL code to target Java code, the tool executes a mature automated conversion and refactoring process by first constructing a comprehensive Intermediate Object Model of the legacy system in an intermediate translation language.
> Once modeled within the tool engine, SMEs employ an iterative process of applying rules and tuning to output the transformed code into the target Java language.
Seems like they manually construct an IR of the source system, then translate the IR to Java Code.
Automatically transpiling COBOL code to Java will lose all possibility of maintenance moving forward. So it is not considered in the first place, nor it is what they need. They need a semi-auto REWRITE of their legacy system, not just target it to Java.
what would have been the total cost of this whole conversion effort COBOL to JAVA system?
even if it $100 millions, it seems a successful project , given DoD mentioned potential savings of $25 millions/year
>Customer Benefits
> For the DoD, the component has been transformed from an expensive mainframe COBOL legacy system to an affordable, modern maintainable Java-based system. All valuable existing business rules have been preserved in the modernized system, while development, test, and production environments were migrated to AWS providing flexibility at reduced cost.
>The DoD’s projected cost savings is $25 million per year, and they are now able to use popular Java programmers to maintain and enhance the critical component.
I see this happening more and more. And for me, it was my guess at why IBM bought RedHat, since the government was likely moving off an IBM solution this allows IBM to still "own" the platform (RHEL + IBM Java) and I presume at some point they will try to own the cloud as well.
It was interesting while I was there helping to transition the Blekko stuff to IBM I brought up how this sort of modernization of their cloud offering was in their strategic interest. IBM is so big, and so complex internally, it was kind of like shouting at a crowd moving through Grand Central station in New York. People can hear you shouting and slowly start to comprehend what you're trying to get them to see. Change is hard in an organization of that size.
> I presume at some point they will try to own the cloud as well.
They already did that, they bought SoftLayer, but it's not working very well since instead of going for "a cloud offering" they are doing a "same kind of ibm offering as usual, but in the cloud !", there is a reason no one is talking about using them on sites like HN their entire target market seems to be company moving from big "enterprise" hosting to big cloud "enterprise" hosting.
It will be interesting to see how this works out, but
I doubt we will hear much publically about it again.
I have been involved in a somwhat similar project where
we had a large system written in an obsolete language.
It was HUGE. And it ran fine. With millions of transaction.
But it required expensive hardware to run it.
The decision was made to convert it to Java.
The effort took years, and the rend result was not
impressive I thought.
In a lot of ways Java was a step down in abstraction
from the more DSL original langauge.
Implementing new rules took more time, and
running it took a lot more orchtecstration.
I am not convinced that it would have been better
to hire and train a few devs in the old system
and let it keep doing its thing for a few decades more.
The article implies it wasn't a very nice or well-structured codebase, so I guess they didn't make it much worse.
And they ripped out the COBOL Data management system and replaced it with a RDBMS. This alone probably is worth it since you can hook it up to business intelligence tools now to create reports and dashboards.
The old language was domain specific, of course it was a better set of abstractions for what you care about.
But maintaining and extending the thing was only going to get more expensive. And once you've paid the price to modernize, you've bought yourselves a lot of savings for 1-2 decades, maybe more.
COBOL is a very niche language that is very well specialized to banking. I don't envision applications rewritten in something else easily for any significant benefit.
I am impressed with this solution, insofar as was described in the press release. But I also have concerns.
My concerns mainly lie with choosing such a proprietary solution over one using more open-source standards. Both in the language selected, along with the backend (Oracle and AWS) being used.
Will there or could there be problems in the future, should the need arise to migrate off one or more of those proprietary platforms?
What will or could happen if there is a security issue that affects or targets AWS, Java, or this system in particular - are we "stuck" again with a potentially "broken" system that can't be easily moved because of vendor lock-in of sorts?
Should Oracle make further moves in the direction of "closing" Java - will programmers continue to learn the language and support it, or will they move to other, more open, solutions?
What happens if or when AWS or Amazon ceases to exist?
I just wonder if we haven't traded a largely unwieldy but mostly known issue of a mainframe and COBOL, for a seemingly known (but questionable) issue of AWS, Java, and Oracle.
Will we just revisit this whole problem again in 50 years?
If so, maybe then we'll make a better decision to use more open and auditable solutions (but I'm not going to hold my breath to that)...
should the need arise to migrate off one or more of those proprietary platforms
Fair point given how things usually go in the private sector. In the public sector, especially DoD, the rules of the game are changed.
What happens if or when AWS or Amazon ceases to exist?
Well guess what, they can't. It's now a national security issue so the necessary services will continue to be provided by a KTLO crew or DoD would facilitate a transaction where another company could absorb the needed technology and be contracted to support it. DoD is the biggest company on earth. FY 2019 budget is $686B.
DoD has a pattern of keeping mission critical end-of-life systems running and supported.
I'm currently advising the US Navy on moving some COBOL systems to Java. Those systems have been lovingly supported by lifelong Navy civilian employees for 40-50 years. Those people are now in retirement age and so they're "replatforming" to Java.
There really is no corollary in the private sector. The default for government systems has been the people who built it stick around and maintain it until they retire. For DoD the biggest risk isn't old technologies and changes in the private sector, it's brain drain when people retire.
You might ask yourself, what 20-something wants to work for the US Navy, as a civilian, and work on a Java implementation (ported from COBOL) for the rest of their career? Well, I'd like to introduce you to them. These guys could work anywhere but they like the Navy. They like that their work matters, it's important, and it's stable. The pay is below market but it isn't terrible. Keep in mind DoD isn't all in DC, many offices are in inexpensive places to live.
Frankly I'd love to work on systems with the meaningful impact that theirs have (non-combat in this case). Read about 18F and what it was like for the people who joined and how they stuck with it anyway.
Nothing is really future proof, though. If it makes if 5 years before any major changes, then it will be good. I don't disagree that eventually this could lead to the same kind of vendor lock-in, but, it is also performant, has support, and will work tomorrow with off the shelf parts. of course you could go with OSS, and while that might be a good technical solution, it doesn't necessarily make it a good business decision. AWS is available now, and its performance characteristics are well understood.
There's more here too. Amazon isn't "American" even though it's CEO is American and has many American employees. Multinational corporations owe no loyalty to anyone other than their shareholders, I mean, Amazon the corporation paid nothing in federal taxes last year. What happens when their bottom line contradicts keeping this country safe?
This underlines how so much of the work and contracts at the DoD don't really revolve around defending the US than being another way to hand out tax payer dollars to already well off corporations.
Considering amazon did all the I feel like the AWS target is reasonable. I suspect that the next time this sort of works needs to be done it will be easier than this round was.
The first one has lots of merits, as it allows for a progressive replacement of COBOL functionalities
The second one immediately lowers the TCO for obvious licensing reasons but does not plan for the future, particularly as COBOL developers retire and thus the resources are scarce.
(Anecdotally the customer chose both: recompilation & progressive replacement by Java code for core assets and emulation for "dead", non core, assets dev & staging environments)
Here's the main problem with this: Where are the requirements?
The COBOL doesn't have automated tests. When you port the COBOL to Java (whether automated or manually) you aren't asking "where are the original requirements?". Contrast this with a "re-write". When you do a re-write you have to go back to the customer and whatever documentation you can find and figure out and re-document what the system should do. This can be far removed from what the system did in its original design docs, and can differ still from what people believe.
This step of codifying tacit knowledge of the behaviour is important in the evolution of systems.
> The COBOL doesn't have automated tests. When you port the COBOL to Java (whether automated or manually) you aren't asking "where are the original requirements?".
You should be asking where is the current documentation of the system specs, and in some cases you might even get something reasonably current and complete when you ask that, depending on how to use the change management process has been.
> When you do a re-write you have to go back to the customer and whatever original documentation you have lying around and you have to figure out and document what the system does.
Both what it does and (and this is where the customer is involved) what it is currently supposed to do.
The system is the specification. Working software that is 65 years old is an organic and evolved entity that can't be manually ported or updated.
Another possible target for this is the IPPS/OPPS Medicaid/Medicare code calculator system - which was written in COBOL for IBM 370 by mostly a single developer. Sadly, the developer passed away about 10 years ago, and the system has yet to be updated. The code was open sourced in a bid to gain assistance for updating from the community. Automation such as this would help significantly.
I'm surprised that automatically translated Java code would be easier to maintain than human-written COBOL. Doesn't it become difficult to understand? I suppose 50 years old COBOL is basically as cryptic as auto-generated Java...
Very interesting write up, especially for what one might expect to be straight up promotional piece. It's nice to hear that amazon can guarantee the necessary uptime to run a system of such importance.
It'd be interesting to compare this approach to modernization approaches in the government. For example the veterans administration is moving away from their ancient but popular and decently well regarded electronic health record system Vista (https://en.wikipedia.org/wiki/VistA) to cerner.
My Father works as an engineer for the VA, the hidden costs of this transition is the need to rewire almost the entire VA hospital system to comply with cerner networking requirements at the cost of $100's of millions per hospital before the software can begin to be deployed.
The other hidden cost is that these ancient software systems are extremely efficient in terms of cpu / memory utilization compared to modern equivalents so the hardware requirements go up considerably which impacts power / energy efficiency and direct deployment costs for replacement systems. Not saying modernization is not a win in the long term but there are direct and indirect costs with these big transitions.
I wouldn't be surprised if Amazon is a national security component now and that may be why they don't pay taxes are get have deliveries subsidized etc. Can the government let them fail if they are critical infrastructure?
At a wild guess, I’d imagine getting something working probably took about 18 hours, with the rest of the time spent writing tests and defining behaviour of the legacy system to prove that the port actually worked
We migrated the core banking from a small private bank, and that was 12 million COBOL lines. It actually seems like a "small" system.
We used the same approach (java translation). Running the cobol is actually the easy part. The hard part is when you start throwing in managing files the mainframe way, the database (needs to quack like DB2), the transaction monitor (CICS), presentation layer (3270 like), scheduler, etc.
And there is plenty of other technologies usually deployed in such shops (IMS, MQ, etc). And you need to support it all...
> At the end of Phase 1, the Java program is “dirty” with remaining COBOL coding practices. At the end of Phase 2, the Java program is “clean” without COBOL remnants.
Great use to match up with the dirty Java logo. I’m sure they had a lot of fun designing that, and this dirty/clean terminology will probably stick in other source to source transformations.
I've done something like this using syntax-directed translation. One can proceed by changing the transformation granularity. First, you have naive direct transliterations plus scaffolding libraries which result in non-idiomatic target language code. (In the article's case, this would be non-idiomatic machine produced Java) It runs and passes some of the tests.
Next step is to fix the bugs, and get all the tests to run.
The last step, to get to "clean" involves going through the code base, looking for project-specific idioms, then hand-tuning the syntax matching engine with rules that output idiomatic target language code. This might also involve refactorings to make the scaffolding libraries more idiomatic as well.
Since one is coding on the transformation engine, instead of directly on the application itself, production changes and maintenance can proceed, and no code freeze is needed.
What is "architectural future state requirement"? Why didn't the emulation idea work? Isn't that the industry standard for moving old code to new hardware?
I have limited experience with mainframe emulation, but from what I've seen, trying to do it in a live business environment is challenging at best because all of that old mainframe software was written with a lot of assumptions that don't apply to non-mainframe environments. Things like blazing-fast I/O, virtually guaranteed uptime of the mainframe, etc.
As one example, when I was a systems engineer, I got a call from a team that was trying to run a business application from z/OS in emulation on an x86 server. It was running incredibly slowly, even on a fairly beefy server. I looked in its data directory, and there were hundreds of thousands of individual files in a single directory. Not a tree. One directory. That might work fine on a mainframe, but just doing a dir or ls (depending on the host OS) took something like ten minutes.
Similarly, I remember some folks being dumbfounded at the idea that their batch processes needed error-handling when they were doing things that involved talking over the network, because when everything was running within that one giant IBM system, failures were pretty rare.
Those both sound like they should be easy to tack on when running in an emulator, but they're just the tip of the iceberg. When you have code that's 30-50 years old, written for a vastly different platform, there are going to be a lot of gotchas like that.
The authors emphasized in multiple places that moving off COBOL was a first-order project requirement.
I did find it interesting that the original system, and delivered overhaul, both ran Oracle databases. Overall transactional volume is low (500k/day). Perhaps the DoD has significant investments in stored procedures? Or perhaps Aurora is not ready for prime time in AWS Gov Cloud? (It’s no secret that AWS really struggles with keeping feature parity between Gov and commercial.)
Amazon, and AWS in particular, sure seems all in on building technology for the military and police.
Anyone else experience some cognitive dissonance in "the world's most customer centric company" also someday powering the facial recognition systems used by customs, border patrol, police, and military systems?
It seems ripped straight of a cyberpunk novel to me.
My main question is why this can't be done incrementally? The system can almost always be broken down into functional areas? Does Agile here just not apply?
[+] [-] kyberias|7 years ago|reply
Let us stop and give some caring thoughts to the people who will maintain that code base.
[+] [-] tanilama|7 years ago|reply
> Once modeled within the tool engine, SMEs employ an iterative process of applying rules and tuning to output the transformed code into the target Java language.
Seems like they manually construct an IR of the source system, then translate the IR to Java Code.
Automatically transpiling COBOL code to Java will lose all possibility of maintenance moving forward. So it is not considered in the first place, nor it is what they need. They need a semi-auto REWRITE of their legacy system, not just target it to Java.
[+] [-] LrnByTeach|7 years ago|reply
even if it $100 millions, it seems a successful project , given DoD mentioned potential savings of $25 millions/year
>Customer Benefits
> For the DoD, the component has been transformed from an expensive mainframe COBOL legacy system to an affordable, modern maintainable Java-based system. All valuable existing business rules have been preserved in the modernized system, while development, test, and production environments were migrated to AWS providing flexibility at reduced cost.
>The DoD’s projected cost savings is $25 million per year, and they are now able to use popular Java programmers to maintain and enhance the critical component.
[+] [-] Teckla|7 years ago|reply
[+] [-] ChuckMcM|7 years ago|reply
It was interesting while I was there helping to transition the Blekko stuff to IBM I brought up how this sort of modernization of their cloud offering was in their strategic interest. IBM is so big, and so complex internally, it was kind of like shouting at a crowd moving through Grand Central station in New York. People can hear you shouting and slowly start to comprehend what you're trying to get them to see. Change is hard in an organization of that size.
[+] [-] nolok|7 years ago|reply
They already did that, they bought SoftLayer, but it's not working very well since instead of going for "a cloud offering" they are doing a "same kind of ibm offering as usual, but in the cloud !", there is a reason no one is talking about using them on sites like HN their entire target market seems to be company moving from big "enterprise" hosting to big cloud "enterprise" hosting.
And they're not buying AWS any time soon.
[+] [-] Ixiaus|7 years ago|reply
[+] [-] ThinkBeat|7 years ago|reply
I have been involved in a somwhat similar project where we had a large system written in an obsolete language. It was HUGE. And it ran fine. With millions of transaction. But it required expensive hardware to run it.
The decision was made to convert it to Java. The effort took years, and the rend result was not impressive I thought.
In a lot of ways Java was a step down in abstraction from the more DSL original langauge. Implementing new rules took more time, and running it took a lot more orchtecstration.
I am not convinced that it would have been better to hire and train a few devs in the old system and let it keep doing its thing for a few decades more.
[+] [-] iSnow|7 years ago|reply
And they ripped out the COBOL Data management system and replaced it with a RDBMS. This alone probably is worth it since you can hook it up to business intelligence tools now to create reports and dashboards.
[+] [-] lallysingh|7 years ago|reply
But maintaining and extending the thing was only going to get more expensive. And once you've paid the price to modernize, you've bought yourselves a lot of savings for 1-2 decades, maybe more.
[+] [-] user5994461|7 years ago|reply
COBOL is a very niche language that is very well specialized to banking. I don't envision applications rewritten in something else easily for any significant benefit.
[+] [-] cr0sh|7 years ago|reply
My concerns mainly lie with choosing such a proprietary solution over one using more open-source standards. Both in the language selected, along with the backend (Oracle and AWS) being used.
Will there or could there be problems in the future, should the need arise to migrate off one or more of those proprietary platforms?
What will or could happen if there is a security issue that affects or targets AWS, Java, or this system in particular - are we "stuck" again with a potentially "broken" system that can't be easily moved because of vendor lock-in of sorts?
Should Oracle make further moves in the direction of "closing" Java - will programmers continue to learn the language and support it, or will they move to other, more open, solutions?
What happens if or when AWS or Amazon ceases to exist?
I just wonder if we haven't traded a largely unwieldy but mostly known issue of a mainframe and COBOL, for a seemingly known (but questionable) issue of AWS, Java, and Oracle.
Will we just revisit this whole problem again in 50 years?
If so, maybe then we'll make a better decision to use more open and auditable solutions (but I'm not going to hold my breath to that)...
[+] [-] ryanmarsh|7 years ago|reply
Fair point given how things usually go in the private sector. In the public sector, especially DoD, the rules of the game are changed.
What happens if or when AWS or Amazon ceases to exist?
Well guess what, they can't. It's now a national security issue so the necessary services will continue to be provided by a KTLO crew or DoD would facilitate a transaction where another company could absorb the needed technology and be contracted to support it. DoD is the biggest company on earth. FY 2019 budget is $686B.
DoD has a pattern of keeping mission critical end-of-life systems running and supported.
I'm currently advising the US Navy on moving some COBOL systems to Java. Those systems have been lovingly supported by lifelong Navy civilian employees for 40-50 years. Those people are now in retirement age and so they're "replatforming" to Java.
There really is no corollary in the private sector. The default for government systems has been the people who built it stick around and maintain it until they retire. For DoD the biggest risk isn't old technologies and changes in the private sector, it's brain drain when people retire.
You might ask yourself, what 20-something wants to work for the US Navy, as a civilian, and work on a Java implementation (ported from COBOL) for the rest of their career? Well, I'd like to introduce you to them. These guys could work anywhere but they like the Navy. They like that their work matters, it's important, and it's stable. The pay is below market but it isn't terrible. Keep in mind DoD isn't all in DC, many offices are in inexpensive places to live.
Frankly I'd love to work on systems with the meaningful impact that theirs have (non-combat in this case). Read about 18F and what it was like for the people who joined and how they stuck with it anyway.
[+] [-] abakker|7 years ago|reply
[+] [-] noobermin|7 years ago|reply
This underlines how so much of the work and contracts at the DoD don't really revolve around defending the US than being another way to hand out tax payer dollars to already well off corporations.
[+] [-] DeepYogurt|7 years ago|reply
[+] [-] dangerboysteve|7 years ago|reply
https://federalnewsnetwork.com/tom-temin-commentary/2018/01/...
edit: Added link to patent application.
http://www.freepatentsonline.com/y2018/0253287.html
[+] [-] karambahh|7 years ago|reply
Recompiling the COBOL codebase on Linux and accessing and loading it as shared objects (via JNI for JVM usage)
Running Hercules ( https://en.wikipedia.org/wiki/Hercules_%28emulator%29 ), an IBM mainframe emulator
The first one has lots of merits, as it allows for a progressive replacement of COBOL functionalities
The second one immediately lowers the TCO for obvious licensing reasons but does not plan for the future, particularly as COBOL developers retire and thus the resources are scarce.
(Anecdotally the customer chose both: recompilation & progressive replacement by Java code for core assets and emulation for "dead", non core, assets dev & staging environments)
[+] [-] ryanmarsh|7 years ago|reply
The COBOL doesn't have automated tests. When you port the COBOL to Java (whether automated or manually) you aren't asking "where are the original requirements?". Contrast this with a "re-write". When you do a re-write you have to go back to the customer and whatever documentation you can find and figure out and re-document what the system should do. This can be far removed from what the system did in its original design docs, and can differ still from what people believe.
This step of codifying tacit knowledge of the behaviour is important in the evolution of systems.
[+] [-] dragonwriter|7 years ago|reply
You should be asking where is the current documentation of the system specs, and in some cases you might even get something reasonably current and complete when you ask that, depending on how to use the change management process has been.
> When you do a re-write you have to go back to the customer and whatever original documentation you have lying around and you have to figure out and document what the system does.
Both what it does and (and this is where the customer is involved) what it is currently supposed to do.
[+] [-] binarymax|7 years ago|reply
Another possible target for this is the IPPS/OPPS Medicaid/Medicare code calculator system - which was written in COBOL for IBM 370 by mostly a single developer. Sadly, the developer passed away about 10 years ago, and the system has yet to be updated. The code was open sourced in a bid to gain assistance for updating from the community. Automation such as this would help significantly.
[+] [-] tinix|7 years ago|reply
[+] [-] jefft255|7 years ago|reply
[+] [-] jrowley|7 years ago|reply
It'd be interesting to compare this approach to modernization approaches in the government. For example the veterans administration is moving away from their ancient but popular and decently well regarded electronic health record system Vista (https://en.wikipedia.org/wiki/VistA) to cerner.
https://www.hitechanswers.net/va-establishes-office-of-ehr-m...
[+] [-] jakebol|7 years ago|reply
The other hidden cost is that these ancient software systems are extremely efficient in terms of cpu / memory utilization compared to modern equivalents so the hardware requirements go up considerably which impacts power / energy efficiency and direct deployment costs for replacement systems. Not saying modernization is not a win in the long term but there are direct and indirect costs with these big transitions.
[+] [-] edoo|7 years ago|reply
[+] [-] tgraham|7 years ago|reply
[+] [-] darkr|7 years ago|reply
[+] [-] fork1|7 years ago|reply
[+] [-] wycy|7 years ago|reply
[+] [-] maccam94|7 years ago|reply
[+] [-] jxramos|7 years ago|reply
Great use to match up with the dirty Java logo. I’m sure they had a lot of fun designing that, and this dirty/clean terminology will probably stick in other source to source transformations.
[+] [-] stcredzero|7 years ago|reply
Next step is to fix the bugs, and get all the tests to run.
The last step, to get to "clean" involves going through the code base, looking for project-specific idioms, then hand-tuning the syntax matching engine with rules that output idiomatic target language code. This might also involve refactorings to make the scaffolding libraries more idiomatic as well.
Since one is coding on the transformation engine, instead of directly on the application itself, production changes and maintenance can proceed, and no code freeze is needed.
[+] [-] craftyguy|7 years ago|reply
(this rarely works out well for taxpayers in the long term..)
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] tantalor|7 years ago|reply
[+] [-] blincoln|7 years ago|reply
As one example, when I was a systems engineer, I got a call from a team that was trying to run a business application from z/OS in emulation on an x86 server. It was running incredibly slowly, even on a fairly beefy server. I looked in its data directory, and there were hundreds of thousands of individual files in a single directory. Not a tree. One directory. That might work fine on a mainframe, but just doing a dir or ls (depending on the host OS) took something like ten minutes.
Similarly, I remember some folks being dumbfounded at the idea that their batch processes needed error-handling when they were doing things that involved talking over the network, because when everything was running within that one giant IBM system, failures were pretty rare.
Those both sound like they should be easy to tack on when running in an emulator, but they're just the tip of the iceberg. When you have code that's 30-50 years old, written for a vastly different platform, there are going to be a lot of gotchas like that.
[+] [-] posixplz|7 years ago|reply
I did find it interesting that the original system, and delivered overhaul, both ran Oracle databases. Overall transactional volume is low (500k/day). Perhaps the DoD has significant investments in stored procedures? Or perhaps Aurora is not ready for prime time in AWS Gov Cloud? (It’s no secret that AWS really struggles with keeping feature parity between Gov and commercial.)
[+] [-] taftster|7 years ago|reply
> "Difficulty finding qualified, affordable COBOL programmers."
Continuing to emulate the COBOL environment doesn't quite solve the problem that finding COBOL developers is becoming harder and harder.
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] Pfhreak|7 years ago|reply
Anyone else experience some cognitive dissonance in "the world's most customer centric company" also someday powering the facial recognition systems used by customs, border patrol, police, and military systems?
It seems ripped straight of a cyberpunk novel to me.
[+] [-] nartz|7 years ago|reply