Amazon EMR announces EMR release 6.0.0 with new major versions of Hadoop, Hive, HBase, Amazon Linux 2, and support for Docker

Hive users can now use Hive Live Long and Process (LLAP) with EMR release 6.0.0, providing a 2x performance speedup over EMR 5.29 with up to 10x improvement on individual Hive TPC-DS queries*. Hive LLAP is a new execution model in Hive that uses persistent daemons with dynamic in-memory caching to speed up query execution. … Read more Amazon EMR announces EMR release 6.0.0 with new major versions of Hadoop, Hive, HBase, Amazon Linux 2, and support for Docker

Amazon ECS and AWS Fargate support for Amazon EFS File Systems now generally available

Applications like WordPress, Drupal, and JIRA use EFS to store user data and application state, with active/standby container pairs running in different availability zones for high availability. Data science teams use EFS to store shared training data and home directories, allowing notebook containers and distributed training jobs to access the data in parallel.  To use … Read more Amazon ECS and AWS Fargate support for Amazon EFS File Systems now generally available

Dealing with Growing Impatience? Push your Real-Time ML Services to Production on AWS!

Building the right cloud infrastructure for your ML application is usually underrated. Here is a glimpse of how easy it can get thanks to widely democratized cloud provisioning. The number of times I have seen research groups, students or coworkers coming up with a model achieving great performances has been recently astonishing. (And I’m not … Read more Dealing with Growing Impatience? Push your Real-Time ML Services to Production on AWS!

Amazon Managed Cassandra Service (preview) now helps you coordinate increments and decrements to column values by using counters

Counters make it easier to coordinate increments and decrements to column values in distributed systems, or in scenarios where increments happen rapidly. For example, you can use counters to track the number of entries in a log file or the number of times a post has been viewed on a social network. Using counters ensures … Read more Amazon Managed Cassandra Service (preview) now helps you coordinate increments and decrements to column values by using counters

Amplify CLI adds support for additional Lambda runtimes (Java, Go, .NET and Python) and Lambda cron jobs

In addition, Amplify CLI can schedule recurring Lambda functions to execute periodically (e.g every minute, hourly, daily, monthly or yearly basis). You can also use custom AWS cron expressions to describe more complex schedules such as “10:15 AM on the last Friday of every month”. Read our blog post to learn how to add recurring … Read more Amplify CLI adds support for additional Lambda runtimes (Java, Go, .NET and Python) and Lambda cron jobs

AWS Serverless Application Repository allows sharing public applications from all commercial AWS regions

Previously, application authors could publish public applications only from one of the master regions, us-east-1 or us-east-2. If they had a different default or preferred region, they had to either keep their application private, or switch to a master region to publish it. With this launch, they have the convenience to publish public apps from … Read more AWS Serverless Application Repository allows sharing public applications from all commercial AWS regions

AWS IoT Device Defender Now Available in the AWS China (Beijing) Region, operated by Sinnet, and the AWS China (Ningxia) Region, operated by NWCD

AWS IoT Device Defender is now available in the AWS China (Beijing) Region, operated by Sinnet and the AWS China (Ningxia) Region, operated by NWCD. AWS IoT Device Defender is a fully managed service that helps you secure your fleet of IoT devices. It continuously audits device configurations for security vulnerabilities. It also continuously monitors … Read more AWS IoT Device Defender Now Available in the AWS China (Beijing) Region, operated by Sinnet, and the AWS China (Ningxia) Region, operated by NWCD

AWS RoboMaker now supports configurable Simulation Unit limit

A single SU provides 1 vCPU and 2 GB of memory, and the number of SUs required to run your simulation job is determined by the maximum resource requirement between vCPU and memory. This new capability enables you to configure the maximum amount of SUs that can be used by your simulation job so that … Read more AWS RoboMaker now supports configurable Simulation Unit limit

Amazon GameLift in 2020 – Major update now available in preview

That’s why today we’re excited to announce the release of an update that greatly enhances the flexibility in how you can adopt and use GameLift. Available now in preview, the updated GameLift FleetIQ feature allows you to onboard server workloads to the cloud at your own pace, partially or completely, using your own tools independent … Read more Amazon GameLift in 2020 – Major update now available in preview

Amazon Elastic File System announces 400% increase in read operations for General Purpose mode file systems

General Purpose mode (GP mode) is the default performance mode for Amazon EFS. It offers the lowest per-operation latency and is the recommended choice for most applications. Amazon EFS also offers the Max I/O performance mode which can scale to higher levels of aggregate throughput and supports over 500,000 operations per second with slightly higher … Read more Amazon Elastic File System announces 400% increase in read operations for General Purpose mode file systems

AWS Storage Gateway adds audit logs for File Gateway to address enterprise compliance requirements

File Gateway provides on-premises applications with file-based, cached access to virtually unlimited cloud storage using SMB and NFS protocols. With this launch, SMB client operations for files and folders are logged to provide IT administrators and compliance managers with the information they need about user access to files and folders. Key operations logged for files … Read more AWS Storage Gateway adds audit logs for File Gateway to address enterprise compliance requirements

AWS Canada (Central) Region Adds Third Availability Zone

This third AZ provides customers with additional flexibility to architect scalable, fault-tolerant, and highly available applications in AWS Canada (Central) Region, and will support additional AWS services in Canada. This launch brings AWS’s global AZ total to 70 AZs within 22 geographic Regions around the world. See our newly released blog.   Availability Zones located … Read more AWS Canada (Central) Region Adds Third Availability Zone

Amazon FSx for Windows File Server is Now Available in the AWS Asia Pacific (Hong Kong) Region

Amazon FSx for Windows File Server provides fully managed, highly reliable file storage that is accessible over the industry-standard Service Message Block (SMB) protocol. It is built on Windows Server, delivering a wide range of administrative features such as user quotas, end-user file restore, and Microsoft Active Directory (AD) integration. It offers single-AZ and multi-AZ … Read more Amazon FSx for Windows File Server is Now Available in the AWS Asia Pacific (Hong Kong) Region

Amazon FSx for Lustre is Now Available in the AWS Asia Pacific (Hong Kong) Region

Amazon FSx for Lustre provides a high-performance file system optimized for fast processing of workloads such as machine learning, high performance computing (HPC), video processing, financial modeling, and electronic design automation (EDA). Amazon FSx for Lustre file systems can be configured as standalone file systems or can be linked to an Amazon S3 bucket. When … Read more Amazon FSx for Lustre is Now Available in the AWS Asia Pacific (Hong Kong) Region

Why we don’t use Lambda for serverless machine learning

AWS’s serverless compute platform doesn’t work for ML inference Source: Pexels When it comes to serving model predictions, serverless has become the popular architecture. Most ML deployments can be conceptualized as a straightforward predict(), and if—from the developer’s perspective—a model is essentially just a function, it makes perfect sense to deploy it on a serverless … Read more Why we don’t use Lambda for serverless machine learning

AWS App Mesh adds support to connect services deployed in multiple AWS accounts into a shared mesh

Customers may deploy their services in separate AWS accounts for reasons of security, cost allocation or alignment with team boundaries. However, they also want to represent logical application boundaries and simplify configuration between connected services with a single mesh. Until now, AWS App Mesh was scoped to an account boundary. Starting today, customers can use … Read more AWS App Mesh adds support to connect services deployed in multiple AWS accounts into a shared mesh

New Quick Start deploys AWS Database Migration Service Automation Framework

This Amazon Web Services (AWS) Quick Start deploys AWS Database Migration Service (AWS DMS) Automation Framework on the AWS Cloud in about 30 minutes. This Quick Start is for users who currently do one of the following:  Separate AWS DMS migration tasks into full-load and change-data-capture (CDC) phases Require a fully automated code deployment framework … Read more New Quick Start deploys AWS Database Migration Service Automation Framework

Amazon Managed Blockchain now supports Amazon CloudWatch Logs

Hyperledger Fabric emits peer node, chaincode and CA logs. Peer node logs help you debug timeout errors associated with proposals and rejected proposals that do not meet the endorsement policies. Chaincode logs contain the results of instantiating, invoking and querying chaincode, and helps to analyze and debug the business logic. CA logs help you determine … Read more Amazon Managed Blockchain now supports Amazon CloudWatch Logs

Building a k-NN Similarity Search Engine using Amazon Elasticsearch and SageMaker

Step-by-Step Guide to Building Efficient and Scalable Document Similarity Search Engine Photo by NeONBRAND on Unsplash Amazon Elasticsearch Service recently added support for k-nearest neighbor search. It enables you to run high scale and low latency k-NN search across thousands of dimensions with the same ease as running any regular Elasticsearch query. k-NN similarity search … Read more Building a k-NN Similarity Search Engine using Amazon Elasticsearch and SageMaker

Amazon CloudWatch dashboards metric quotas has increased

CloudWatch dashboards provide operational visibility on the health and performance of your resources for monitoring and troubleshooting. To define a dashboard, you can select individual metrics or search for up to 500 metrics at a time. When combined with CloudWatch metric math, you can sum or sort the metrics and display the top N results … Read more Amazon CloudWatch dashboards metric quotas has increased

Amazon CloudWatch now supports AWS PrivateLink in AWS GovCloud (US) Regions

Amazon CloudWatch now supports AWS PrivateLink in the AWS GovCloud (US) Regions. This allows you to privately connect to CloudWatch Metrics and Logs, securely on the AWS network. AWS PrivateLink allows you to privately access services hosted on AWS in a highly available and scalable manner, without using public IPs and without requiring the traffic … Read more Amazon CloudWatch now supports AWS PrivateLink in AWS GovCloud (US) Regions

Amazon CloudWatch cross-account cross-region dashboards now available in AWS GovCloud (US) Regions

CloudWatch cross-account cross-region dashboards can be used to visualize application or resource health and performance in multiple AWS accounts and Regions. You can share visibility of monitoring data from one or more accounts in AWS GovCloud (US) Regions with one or more monitoring accounts also in AWS GovCloud (US) Regions. This enables you to more … Read more Amazon CloudWatch cross-account cross-region dashboards now available in AWS GovCloud (US) Regions

AWS Site-to-Site VPN now supports certificate authentication for connections to AWS Transit Gateway

To use certificates with your VPN connections, you first create a subordinate Certificate Authority (CA) from AWS Certificate Manager Private Certificate Authority. Then, generate a digital certificate from the CA you created to use on your customer gateway device. When using certificates for authentication, you do not need to specify an IP address for your … Read more AWS Site-to-Site VPN now supports certificate authentication for connections to AWS Transit Gateway

Amazon AppStream 2.0 Now Supports EC2 g4dn Instances

AppStream 2.0 offers six different g4dn instance sizes, ranging from 4 vCPUs and 16 GB of memory to 64 vCPUs and 256 GiB of memory. The following g4dn instance sizes are available for AppStream 2.0: stream.graphics.g4dn.xlargestream.graphics.g4dn.2xlargestream.graphics.g4dn.4xlargestream.graphics.g4dn.8xlargestream.graphics.g4dn.12xlargestream.graphics.g4dn.16xlarge  All instance sizes, except g4dn.12xlarge, offer 16 GB of discrete video memory. G4dn.12xlarge offers 64 GiB offers of GPU … Read more Amazon AppStream 2.0 Now Supports EC2 g4dn Instances

Amazon Managed Cassandra Service (preview) is now available in 18 AWS Regions

With this launch, Amazon MCS is available in preview in the US East (N. Virginia), US East (Ohio), US West (Oregon), US West (N. California), Canada (Central), South America (Sao Paulo), Europe (Ireland), Europe (Frankfurt), Europe (London), Europe (Paris), Europe (Stockholm), Middle East (Bahrain), Asia Pacific (Hong Kong), Asia Pacific (Singapore), Asia Pacific (Tokyo), Asia … Read more Amazon Managed Cassandra Service (preview) is now available in 18 AWS Regions

Amazon Managed Cassandra Service now helps you manage access to your keyspaces and tables by using AWS IAM roles and federated identities

IAM roles can help you manage user and application access to your MCS resources by using temporary security credentials for authentication instead of sharing long-term credentials. For example, you can create an IAM role that grants an application read and write permissions only to specific tables in your account. IAM roles can also be used … Read more Amazon Managed Cassandra Service now helps you manage access to your keyspaces and tables by using AWS IAM roles and federated identities

You now can update your Amazon DynamoDB global tables from version 2017.11.29 to the latest version with a few clicks in the DynamoDB Console

You now can update your global tables from version 2017.11.29 to the latest version of global tables (2019.11.21) with a few clicks in the DynamoDB Console. By upgrading the version of your global tables, you can increase the availability of your DynamoDB tables easily by extending your existing tables into additional AWS Regions, with no … Read more You now can update your Amazon DynamoDB global tables from version 2017.11.29 to the latest version with a few clicks in the DynamoDB Console

Setting Up Jupyter on AWS

A scriptable list of command lines to deploy Jupyter in AWS, securely and cost-effectively, with added exercises Image sources: commons.wikimedia.org and aws.amazon.com In my previous post I gave the equivalent instructions for Setting Up Jupyter on Google Cloud — if you are more into Google Cloud, go and check that article. AWS is one of … Read more Setting Up Jupyter on AWS

Amazon CloudWatch Anomaly Detection now available in the AWS GovCloud (US) Regions

You can apply CloudWatch Anomaly Detection on metrics in your account, including custom and AWS vended metrics. CloudWatch Anomaly Detection will automatically determine a range of expected behavior, which you can optionally customize by specifying data exclusion periods, anomaly sensitivity, and the daylight-saving time zone. You can create alarms to notify you when anomalies occur … Read more Amazon CloudWatch Anomaly Detection now available in the AWS GovCloud (US) Regions

Amazon Athena adds support for managing Athena Workgroups using AWS CloudFormation

AWS CloudFormation templates help customers to provision and configure their AWS resources. Athena Workgroups help customers to isolate different workloads and set up cost controls. Previously, to create or update a workgroup, customers had to either use the Athena console, or Athena APIs. With this release, customers can create and update their Athena Workgroups using … Read more Amazon Athena adds support for managing Athena Workgroups using AWS CloudFormation

API Gateway Offers Private Integrations with AWS ELB and AWS CloudMap as Part of HTTP APIs GA Release

Prior to its general availability release, HTTP APIs enabled customers to build APIs for their serverless applications or to proxy requests to HTTP endpoints. Now, customers can also leverage the benefits of HTTP APIs to build APIs for services behind private ALBs, private NLBs, and IP-based services registered in CloudMap such as ECS tasks. To … Read more API Gateway Offers Private Integrations with AWS ELB and AWS CloudMap as Part of HTTP APIs GA Release

Configure Cloud Environments for Batch ML Jobs on AWS

In this post, I will walk you through the preparation steps that enable you to schedule a containerized machine learning model as a batch job on AWS. It is the second part of this post series, which assumes you have finished the first 2 steps in Part 1 to create a Dockerfile and build a … Read more Configure Cloud Environments for Batch ML Jobs on AWS

HDR to SDR Tone Mapping Now Available with AWS Elemental MediaConvert

Since there isn’t a standard way to map the larger color space and higher luminance levels of HDR into the smaller color space and lower luminance levels of SDR, the HDR to SDR conversion is often performed by re-grading content. The MediaConvert tone mapping process is designed to approximate the outcome of HDR to SDR … Read more HDR to SDR Tone Mapping Now Available with AWS Elemental MediaConvert

Amazon Data Lifecycle Manager (DLM) Adds Support For 1 hour Backup Interval

We launched DLM in July 2018 to enable automation of creation and retention of EBS volume snapshots via policies. Since then, we have added multiple features including copy of tags from source volume to snapshots, CloudFormation support, capability for more frequent backups, cross region copy of snapshots, retention based on age of snapshot. Today, we … Read more Amazon Data Lifecycle Manager (DLM) Adds Support For 1 hour Backup Interval

Streaming Data Analytics with Data Warehouses

Use Amazon Kinesis Data Firehose, Redshift, and QuickSight to Analyze Streaming Data Audio Introduction Databases are ideal for storing and organizing data that requires a high volume of transaction-oriented query processing while maintaining data integrity. In contrast, data warehouses are designed for performing data analytics on vast amounts of data from one or more disparate … Read more Streaming Data Analytics with Data Warehouses

Provision AWS Chatbot configurations with AWS CloudFormation

AWS Chatbot is an interactive agent that makes it easy to monitor and interact with your AWS resources in your Slack channels and Amazon Chime chat rooms. With AWS Chatbot you can receive alerts and run commands to return diagnostic information, invoke AWS Lambda functions, and create AWS support cases so that your team can … Read more Provision AWS Chatbot configurations with AWS CloudFormation

AWS WAF adds Anonymous IP List for AWS Managed Rules

You can now identify and block requests to your web applications from VPN, Tor nodes, proxies, and data centers by using the new Anonymous IP List for AWS Managed Rules. This new managed rule group helps you block users who attempt to hide their identity or evade geographic restrictions. The managed rule group can also … Read more AWS WAF adds Anonymous IP List for AWS Managed Rules

Amazon Managed Blockchain is now available in the Asia Pacific (Singapore) region

Amazon Managed Blockchain is a fully managed service that allows you to set up and manage a scalable blockchain network with just a few clicks. Amazon Managed Blockchain eliminates the overhead required to create the network, and automatically scales to meet the demands of thousands of applications running millions of transactions. Once your network is … Read more Amazon Managed Blockchain is now available in the Asia Pacific (Singapore) region

Amazon Aurora with PostgreSQL compatibility supports in-place upgrade from PostgreSQL 9.6 to 10

PostgreSQL 10 includes various features including native table partitioning, support for improved parallelism in query execution, ICU collation support, column group statistics, enhanced postgres_fdw extension, and many more. Additionally, this release includes updated versions of the PLV8, ip4r, and pg_repack extensions. To upgrade, select the “Modify” option on the AWS Management Console corresponding to the … Read more Amazon Aurora with PostgreSQL compatibility supports in-place upgrade from PostgreSQL 9.6 to 10

AWS CodeCommit Introduces an Open Source Remote Helper

AWS CodeCommit now supports an open-source remote helper for Git, git-remote-codecommit. This package extends Git and supports using authentication for CodeCommit repository access using profile information configured through the AWS CLI. This method supports both IAM users and access through assumed roles, identity providers, and federated access.  Favorite

NoSQL Workbench for Amazon DynamoDB is now generally available

DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. Designing scalable, flexible data models is essential to using nonrelational databases. However, designing data models can be challenging, particularly when building new applications whose data access patterns are not yet known. NoSQL Workbench helps simplify and accelerate the … Read more NoSQL Workbench for Amazon DynamoDB is now generally available

Amazon Athena adds support for querying data in S3 buckets using AWS Identity and Access Management’s aws:CalledVia condition key

When you submit a query to Athena that scans data from Amazon S3, Athena reads the required data from S3 to execute your query on your behalf using SSL encryption. Security conscious customers prefer to apply restrictive policies on their S3 buckets, for example, policy to allow data read access to only whitelisted IP addresses. … Read more Amazon Athena adds support for querying data in S3 buckets using AWS Identity and Access Management’s aws:CalledVia condition key

End-to-end Quantitative Trading — Automating ML using AWSCLI

Employing AWSCLI to spin, run, and tear down an AWS server In a previous post we’ve explored the use of Python in order to obtain historic financial data from an online source (in our case, Yahoo! Finance).We’ve developed two Python scripts; One to grab the data for us, and one to process the data using … Read more End-to-end Quantitative Trading — Automating ML using AWSCLI

Amazon EKS now available in the AWS China (Beijing) Region, operated by Sinnet, and the AWS China (Ningxia) Region, operated by NWCD

Amazon EKS is a managed Kubernetes service that makes it easy for you to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane or worker nodes. Amazon EKS is certified Kubernetes conformant, so existing applications running on upstream Kubernetes are compatible with Amazon EKS. You can also easily … Read more Amazon EKS now available in the AWS China (Beijing) Region, operated by Sinnet, and the AWS China (Ningxia) Region, operated by NWCD