> Our policy for evolving the API over time is that new functionality can be added, while removing or changing existing functionality can only be done if the benefits exceed the (often significant) costs to users and only after giving clear deprecation warnings to those users for at least one year. In general, we encourage changes that improve clarity in the API of the library but strongly discourage breaking backward compatibility, given our position near the base of the scientific Python computing stack.
This is one of the nuggets out of it. I have been using Python with scipy and more extensively with it's sci-kit library since 2014. Started with it by solving discrete traveling sales man problem for solution of sitting arrangements and was astonished with its ease of use of matrix algebra with these libraries and got hooked to it. It helped me to do scientific computing and create a user friendly user interface on web to help end-user to just focus on siting arrangement and conference Organization. Can solve it with Matlab or GNU octave, but couldn’t have developed easily the whole system with user interface in them.
If Swift wanted to be used in scientific computing it needs to follow the similar policy of managing its scientific stack.
Julia is good for scientific computing, but it’s UI and web development libraries are isn’t that powerful. So will see how it develops.
I constantly see on HN criticism of Python and something rewritten in Rust as something revolutionary or ground breaking that it will change the whole world. Earlier this fad was with go language. Probably in time to come Rust will be used for some core library development which brings substantial benefits like scipy, not something rewritten as wrapper around C library with unsafe code which can be done with Python much better.
Rust isn't going to replace Python because it isn't an interactive language that can be used for adhoc analysis, but it will be used to write the libraries that python imports when the existing ones are no longer sufficient for people's needs.
Python isn't criticized as much as it is recognized as the lesser tool for certain categories of development. That's fine because it is a general purpose language with many great uses.
For Swift to be taken seriously, it needs to be actually usable across macOS, GNU/Linux and Windows, and not be something that still resembles the heyday of GNUStep experience.
> Can solve it with Matlab or GNU octave, but couldn’t have developed easily the whole system with user interface in them.
This is a very key point. I still find that Matlab often has a larger set of features and better efficiency for linear algebra. But thanks to Scipy, you really can expand your mind about what kinds of systems to interface with.
I agree this is a great philosophy. I have to point out as well, though, that one of the major objections to C++ is that it is "too big" and has too many "dangerous" constructs, both of which are a consequence of this approach.
As a developer using the language this is great: I can write using clean modern approaches* and still use older packages. (Making this all continue to work is a burden on the implementation developers of course.)
* not to argue which modern features are clean or not -- this is about SciPy.
> Probably in time to come Rust will be used for some core library development which brings substantial benefits like scipy, not something rewritten as wrapper around C library with unsafe code which can be done with Python much better.
I think a nice situation would be to have all the core scientific libraries and tools written in rust with clean APIs, independent of any particular scripting language but with the clear goal of providing a clean API to wrapper languages. Then any scripting language can wrap these libs and go - essentially something like a universal numpy/scipy/scikit, etc. This would mean Ruby/Python/JavaScript/Julia, etc. would all be able to get to state of the art immediately while minimizing the duplication of effort across languages and creating a standard "ML" or "Scientific" API.
That discrete traveling salesman problem of seating arrangements ... how did you model that? I have a similar problem with organizing committees where committees have members that are in other committees and so committees with overlapping members can’t meet together etc etc.
I do not see it that way. A whole generation of students is growing with the wrong impression that writing a "for" loop is inevitably inefficient. Also, they believe that it is OK for large arrays of numbers not to be a core language construct, requiring an external library like numpy. These two incorrect beliefs are incredibly damaging in my view.
In a sane programming environment (e.g., with jit compilation), writing a matrix product using three loops should be just as efficient than calling a matrix product routine. The matrix product should also be rightly available without need to "import" anything.
I remember going to the first SciPy conference at Caltech in 2002. IIRC Most of the talk was about needing better package management. What happened since is just amazing.
Good that they got a paper in Nature. It's important such accomplishments get rewarded and acknowledged in the traditional academic way. I hope these people pile up citations. The engineers who build the foundations of modern scientific research get far less credit than they deserve.
I learned about SciPy via Numerical Python (Apress/GitHub: https://github.com/apress/numerical-python). That was a ton of fun, especially after I recovered from freshman calculus flashbacks.
I'm ignorant of numerical computing, so I have to ask: is Fortran still used because it is that much faster than C or any other specialized language? I assume that is the case because I'm sure someone would have rewritten those routines by now otherwise.
Fortran still is the language for numerical computing, most algorithms in this area have been implemented in Fortran. And this is very specialized code that has been test by experts in their areas. That's why a project like SciPy cannot simply replace it with something written in C or Python.
The article appears to have been written in 2018, and just published recently. From the Discussion section:
> SciPy has a strong developer community and a massive user base. GitHub traffic metrics report roughly 20,000 unique visitors to the source website between 14 May 2018 and 27 May 2018 (near the time of writing)
SciPy 1.0 was released towards the end of 2017, so that timing makes sense for a 1.0 retrospective. Not sure why it took so long to get published.
[+] [-] dragonsh|6 years ago|reply
This is one of the nuggets out of it. I have been using Python with scipy and more extensively with it's sci-kit library since 2014. Started with it by solving discrete traveling sales man problem for solution of sitting arrangements and was astonished with its ease of use of matrix algebra with these libraries and got hooked to it. It helped me to do scientific computing and create a user friendly user interface on web to help end-user to just focus on siting arrangement and conference Organization. Can solve it with Matlab or GNU octave, but couldn’t have developed easily the whole system with user interface in them.
If Swift wanted to be used in scientific computing it needs to follow the similar policy of managing its scientific stack.
Julia is good for scientific computing, but it’s UI and web development libraries are isn’t that powerful. So will see how it develops.
I constantly see on HN criticism of Python and something rewritten in Rust as something revolutionary or ground breaking that it will change the whole world. Earlier this fad was with go language. Probably in time to come Rust will be used for some core library development which brings substantial benefits like scipy, not something rewritten as wrapper around C library with unsafe code which can be done with Python much better.
[+] [-] Dowwie|6 years ago|reply
Python isn't criticized as much as it is recognized as the lesser tool for certain categories of development. That's fine because it is a general purpose language with many great uses.
[+] [-] pjmlp|6 years ago|reply
[+] [-] pmarin|6 years ago|reply
The main innovation of recent programming languages is the use of social media for marketing and PR by their companies to promote them.
[+] [-] mxcrossb|6 years ago|reply
This is a very key point. I still find that Matlab often has a larger set of features and better efficiency for linear algebra. But thanks to Scipy, you really can expand your mind about what kinds of systems to interface with.
[+] [-] gumby|6 years ago|reply
As a developer using the language this is great: I can write using clean modern approaches* and still use older packages. (Making this all continue to work is a burden on the implementation developers of course.)
* not to argue which modern features are clean or not -- this is about SciPy.
[+] [-] DataDaoDe|6 years ago|reply
I think a nice situation would be to have all the core scientific libraries and tools written in rust with clean APIs, independent of any particular scripting language but with the clear goal of providing a clean API to wrapper languages. Then any scripting language can wrap these libs and go - essentially something like a universal numpy/scipy/scikit, etc. This would mean Ruby/Python/JavaScript/Julia, etc. would all be able to get to state of the art immediately while minimizing the duplication of effort across languages and creating a standard "ML" or "Scientific" API.
[+] [-] gigatexal|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] warmvanilla|6 years ago|reply
[+] [-] maxnoe|6 years ago|reply
Here is a grand tour of scipy as a jupyternotebook me and a colleague used as intro to scipy for PhDs in astrophysics:
https://github.com/maxnoe/scientific_python_notebooks
You can try it without installing the software by clicking on the binder button.
[+] [-] complex_pi|6 years ago|reply
[+] [-] trentnelson|6 years ago|reply
[+] [-] tobmlt|6 years ago|reply
[+] [-] enriquto|6 years ago|reply
I do not see it that way. A whole generation of students is growing with the wrong impression that writing a "for" loop is inevitably inefficient. Also, they believe that it is OK for large arrays of numbers not to be a core language construct, requiring an external library like numpy. These two incorrect beliefs are incredibly damaging in my view.
In a sane programming environment (e.g., with jit compilation), writing a matrix product using three loops should be just as efficient than calling a matrix product routine. The matrix product should also be rightly available without need to "import" anything.
[+] [-] stelfer|6 years ago|reply
[+] [-] ddavis|6 years ago|reply
[+] [-] timkam|6 years ago|reply
[+] [-] neonate|6 years ago|reply
[+] [-] mud_dauber|6 years ago|reply
[+] [-] valgor|6 years ago|reply
I'm ignorant of numerical computing, so I have to ask: is Fortran still used because it is that much faster than C or any other specialized language? I assume that is the case because I'm sure someone would have rewritten those routines by now otherwise.
[+] [-] coliveira|6 years ago|reply
[+] [-] xianwen|6 years ago|reply
[+] [-] fock|6 years ago|reply
[+] [-] nrclark|6 years ago|reply
[+] [-] kingbirdy|6 years ago|reply
> SciPy has a strong developer community and a massive user base. GitHub traffic metrics report roughly 20,000 unique visitors to the source website between 14 May 2018 and 27 May 2018 (near the time of writing)
SciPy 1.0 was released towards the end of 2017, so that timing makes sense for a 1.0 retrospective. Not sure why it took so long to get published.
[+] [-] geoalchimista|6 years ago|reply
[+] [-] philshem|6 years ago|reply