mmlspark
Microsoft Machine Learning for Apache Spark
microsoft/mmlspark repository overview
Microsoft Machine Learning for Apache Spark
MMLSpark is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. MMLSpark adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.
MMLSpark also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, MMLSpark provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.
Featured Tags
latestdocker pull mcr.microsoft.com/mmlspark/release
How to Use this Image
To launch the container, run the following command:
docker run -it -p 8888:8888 -e ACCEPT_EULA=yes mcr.microsoft.com/mmlspark/release
Navigate to http://localhost:8888/ in your web browser to run the sample notebooks. See the documentation for more on Docker use.
To read the EULA for using the docker image, run
docker run -it -p 8888:8888 mcr.microsoft.com/mmlspark/release eula
Repos
- Key Phrase Extraction: Extracts key phrases to identify the main points. For example, for the input text "The food was delicious and there were wonderful staff", the API returns the main talking points: "food" and "wonderful staff".
- Language Detection: For up to 120 languages, detects which language the input text is written in and report a single language code for every document submitted on the request.
- Azure Cognitive Services Images: Azure Cognitive Services is a set of APIs, SDKs and container images that enables developers to integrate ready-made AI directly into their applications.
Full Tag Listing
Tags go here.
Learn More
-
Read our paper for a deep dive on MMLSpark.
-
See how MMLSpark is used to help endangered species.
-
Explore our collaboration with Apache Spark on image analysis.
Contributing & feedback
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
See CONTRIBUTING.md for contribution guidelines.
To give feedback and/or report an issue, open a GitHub Issue.
Other relevant projects
Apache®, Apache Spark, and Spark® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.
License
- Legal Notice: Container License Information
Tag Summary
No tags have been pushed to this repository yet.