Using Linkerd as a Service Mesh Proxy at WePay
Portworx stack also includes Redis, ElasticSearch, an open source anomaly detection package, andRabbitMQas a message broker separate from OSB.
Using vROps to Right-size Hyper-V VMs
Pivotal Container Service 1.1, Now GA, Helps You Run Kubernetes Without Complexity. Why PKS Just Works.
Venkatesh Sivasubramanian, Data Platform Lead, and Balajee Nagarajan, Director of Software Engineering, GE Predix
Mesosphereis leading the enterprise transformation toward distributed computing and hybrid cloud portability. Mesosphere DC/OS is the premier platform for building, deploying, and elastically scaling modern, containerized applications and big data without compromise. DC/OS makes running containers, data services, and microservices easy, across any infrastructure datacenter or cloud without lock-in.
Portworx enabled a new provisioning model that leveraged Mesos two-level scheduler,formally introduced in August 2016. This way, Predix could introduce a dynamic volume provisioning process that maintains this colossal data store, separately from CF. And Predix can request those services from Portworx usingOpen Service Broker(OSB).
A digest of the weeks most important stories & analyses.Subscribe
Feature image: Pictured above, from left to right: The New Stacks Alex Williams; Venkatesh Sivasubramanian, Data Platform Lead, GE Previx; Balajee Nagarajan, Director of Software Engineering, GE Predix; Goutham Rou, CTO and co-founder, Portworx
The emerging big data architectural pattern
PX-Enterprise: The first containerized storage system to support CSI on Kubernetes
Continuous Security – More on Gartners CARTA Model
EventsInterviewsTechnologyTop StoriesGlobal
Elsewhere, however, customers deploy their own custom-developed tools to Predix Cloud Foundry base platform, for conducting their own predictive analytics on incoming data. Cloud Foundry abstracts the developer experience in a way thats actually sane for them, explained Venkatesh Sivasubramanian, Predix data platform lead. They dont have to manage individual machines, infrastructure, and whatnot. Its actually great for stateless applications. It gives you a nice marketplace you can interact with to create the service instances that you need and bind them to their applications.
We do need to have a single infrastructure that allows us to be able to run both stateless and stateful, he remarked.
As engineers (including those at the IETF) have come to realize, persistent data will always be a necessary resource for both server-side applications and client-side apps. And the need for the Internet of Things (IoT) applications to maintain a reliable, persistent network of connected sensors and apparati, makes a stable and self-contained network state data platform vitally necessary.
Understanding error budget overspend – CRE Life Lessons
Whats more, the CTO said, Portworx is capable of working directly with Mesos, or whatever the scheduler happens to be, to ensure that data-driven applications remain orchestrated sensibly. Predix stack does include Cassandra; in fact, the Predix team created an OSB package forMesosphere DC/OS, for provisioning a Service Broker-compatible version of Cassandra.
How GE Digital Tackled the Stateful / Stateless Problem for Industrial IoT
6 Reasons Why PagerDuty Engineering Stands Out From the Crowd
In parallel, modern applications like Cassandra, MongoDB, or HDFS inherently like to scale out, and handle the scale-out and aggregation themselves, Rou continued. If you take a look at Cassandra, you would want to give it the illusion of direct-attached storage, or this notion of hyperconvergence. Our software architecture fundamentally enables people to run their applications hyperconverged though its more conducive to applications like Cassandra, because you get that low-latency access. Your applications are running closer to where their storage lives.
At the core [of Portworx] is a distributed-block, software-defined storage solution, explained Portworx CTOGoutham Rou, also speaking with Alex Williams. Over the past decade-and-a-half, Rou remarked, adding 10 to 20 terabytes of storage to a network has evolved from architecting a SAN or a NAS over months or even years, to buying a server with 20TB already installed.
Nagarajan leads the team developing Predix as a PaaS platform for use in industrial IoT specifically in managing and utilizing the huge capacities of data generated by machines and connected devices. By deploying custom applications on the Predix platform, industrial customers can manage, monitor, and perform preventative maintenance on their own assets, in so doing, reducing unplanned downtime.
A final thank you from the code Team
In August 2015, General Electric announced its intention to enter the cloud services market. Its original vision involved a product called Predix Cloud, a data gathering and processing platform for its own industrial operations, as well as industrial customers, especially in the fields of healthcare, petroleum, and power generation. Unlike typical public cloud services, Predix Cloud would ingest, interpret, analyze, and report on machine data in real-time.
bosh,cassandra,chef,cloud-foundry,container-storage-interface,ge-digital,ge-predix,general-el,internet-of-things,mesos,mesoscon-2017,mesosphere-dcos,open-service-broker,portworx,smack,stateful-services,
Migrating from CircleCI 1.0 to 2.0: tips for success
Comment on The AppD Approach: How to Monitor Core Apps by The AppD Approach: Deployment Options for .NET Microservices Agent Blog AppDynamics
A new approach to education at Puppet
Translating context: Understanding the global open source community
How GE Digital Tackled the Stateful / Stateless Problem for Industrial IoT
Should You Containerize Your Monolith?
Open Source Data Center (OSDC) Conference 2018 Wrap-Up
For stateless architectures to work efficiently, as the IETF has recently declared inits study of RESTful architecture for the Internet of Things, the messages shared between components must be thorough, complete, and self-contained. Anything a receiving function needs to know about the work it needs to do, must be included within the API call that contacts it.
FutureStack18: Accelerate to the Future
Here is wherepersistent data layer provider Portworxentered the picture. With Predix using Mesos as its scheduler, whatever data layer the team may choose needed to accommodate stateful and stateless applications, just as Mesos does. Predix had been instantiating its Cloud Foundry-facing services using Bosh, along with Terraform to provision infrastructure and Chef to stage the process. But since Cloud Foundry doesnt natively support persistent data clusters, they had to be instantiated outside of Cloud Foundry and then attached using its Service Broker API.
This story was sponsored byMesosphere. TheCloud Foundry Foundationis a sponsor of The New Stack.
Minio private cloud on Pivotal Container Service
Univa Demonstrates Extreme Scale Automation by Deploying More Than One Million Cores in a Single Univa Grid Engine Cluster using AWS
So if the first edition of your industrial IoT platform was built on Cloud Foundry, whichprizes itself for its stateless architecture, how do you reconcile that original vision with, if you will, the real state of things?
RackN: How cloud native is redefining the industry
Aqua Named 2018 Technology Pioneer by the World Economic Forum. Heres Why.
Daniel Hekman of Grape Up Discusses Governments and Cloud Foundry
We are trying to become a cloud-agnostic IoT platform, declared Nagarajan. He described Predix current architecture as relying onHashiCorp Terraformto configure and provision infrastructure on Amazon AWS and Microsoft Azure, with plans for other public cloud providers forthcoming. From there,Boshis used to provisioning Cloud Foundry, which serves as the sole developer-facing component. Customers deploy applications through Cloud Foundry using its familiar model.
Today, lots and lots of companies are running data-rich applications on DC/OS, stated Mesosphere Chief Technology OfficerTobias Knaup[picturedleft],during a MesosCon keynote session. Many of those applications, Knaup said, especially in the IoT space, are running on what its calling the SMACK Stack: Spark, Mesos, Akka, Cassandra, and Kafka. We pulled some numbers: More than 50 percent of all DC/OS clusters are running some of these frameworks. So it truly is a platform for running data-driven applications.
As the colossal quantity of data being ingested became an issue, the Predix team began investigating new options for extending its data storage capabilities. At first, they investigated theContainer Storage Interface (CSI) initiativeunder construction at Cloud Foundry. But asSivasubramanian told The New Stacks Alex Williamsfor anupcoming podcast, CSI had yet to demonstrate signs of developmental maturity of settling down into a prescribed form.
Five Reasons Why Your Microservices Could Be Failing
Hyperledgers Brian Behlendorf Named as Top Blockchain Influencer by New York Times
AnalysisEventsInterviewsPodcastTop StoriesGlobalThe JS Foundation Focuses on Serverless and the Internet of ThingsPlease stay on topic and be respectful of others. Review ourTerms of Use.
How to replicate user errors without the user with Breadcrumbs and Sessions
Squaring the Circle: Making CI/CD Fast and Secure
Alcide Receives SOC 2 Type II Certification
We dont sell or share your email. Occasionally, we send updates and useful info.
Data, Privacy, and Compliance – How We Prepared for GDPR
Now, whatever doubt there may have been among managers in the parent company has vanished, as the engineers plot a way forward.
Ask an Expert: Catching up with Srinath Perera
The biggest challenge that we as platform operators and platform architects [face], Nagarajan told The New Stack, is, when we provide a platform for our end users, we want to make sure that platform is highly available. The way Mesos gives you a unified fabric of compute, Portworx provides a unified fabric of storage. We can then have end users not worry about how to consume an HA application.
Cloud Foundry has this robust marketplace, he said, where you can have asset-building models, analytics-building models. Then Cloud Foundry also provides a very secure interface to Predix itself.
Given the fact that GE has such a huge footprint in many business verticals, Nagarajan continued, it was such a natural, organic growth for us to become the leader in building out this industrial IoT platform, especially because of our inherent domain experience.
Last November, GE expanded this product into the Predix System a network of industrially-targeted, distributed services, complete with applications geared more directly towards its key customer verticals. But even though Predix had thoroughly penetrated the IoT service space, its parent company refrained from using the phrase Internet of Things in Predix promotions.
The way the Predix platform works now, some services are deployed at the edge either on customer premises, or in locations where customers sensors are installed. These services act as local aggregation points, pre-processing sensor data to the extent that it can, prior to its being ingested by the central cloud. In locations where GEs own light sensors are installed, Predix may use GEs data connection tools to perform analytics functions closer to the sensors themselves.
Chef Open Source Community News June 2018
Kubernetes 1.11: Custom Resources, Pod Priority and Preemption, and more
The Most Frequently Asked Questions: A Guide for the Overly Eager New InfluxDB User
We are in the business of dealing with information and we are building the business of dealing with data, whether we like it or not, saidBalajee Nagarajan, the director of software engineering for thePredix industrial IoT platformservice at GE Digital, speaking at the recentMesosConconference in Los Angeles.
If its not exactly a marriage of statefulness and statelessness, it is, for now, a workable cohabitation. Before too long, the company that first turned on the light bulb may recognize the full scope of the idea is set in motion.
