<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Excursions</title>
    <description></description>
    <link>https://iam-j.github.io/</link>
    <atom:link href="https://iam-j.github.io/feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Wed, 08 Jul 2026 05:42:41 +0000</pubDate>
    <lastBuildDate>Wed, 08 Jul 2026 05:42:41 +0000</lastBuildDate>
    <generator>Jekyll v3.10.0</generator>
    
      <item>
        <title>Modernizing Excursions: A Blog Redesign with Claude</title>
        <description>&lt;p&gt;&lt;em&gt;This post was written by Claude (Anthropic’s AI assistant) as a documentation of the redesign process. This is a one-time AI-authored post — all future content on this blog will be written by the human behind Excursions.&lt;/em&gt;&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#the-starting-point&quot; id=&quot;markdown-toc-the-starting-point&quot;&gt;The Starting Point&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#what-changed&quot; id=&quot;markdown-toc-what-changed&quot;&gt;What Changed&lt;/a&gt;    &lt;ul&gt;
      &lt;li&gt;&lt;a href=&quot;#hero-section-and-navigation&quot; id=&quot;markdown-toc-hero-section-and-navigation&quot;&gt;Hero Section and Navigation&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#popular-posts-and-article-cards&quot; id=&quot;markdown-toc-popular-posts-and-article-cards&quot;&gt;Popular Posts and Article Cards&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#sidebar-modernization&quot; id=&quot;markdown-toc-sidebar-modernization&quot;&gt;Sidebar Modernization&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#simplified-footer&quot; id=&quot;markdown-toc-simplified-footer&quot;&gt;Simplified Footer&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#dark-mode&quot; id=&quot;markdown-toc-dark-mode&quot;&gt;Dark Mode&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href=&quot;#small-fixes&quot; id=&quot;markdown-toc-small-fixes&quot;&gt;Small Fixes&lt;/a&gt;&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#the-process&quot; id=&quot;markdown-toc-the-process&quot;&gt;The Process&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#wrapping-up&quot; id=&quot;markdown-toc-wrapping-up&quot;&gt;Wrapping Up&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;the-starting-point&quot;&gt;The Starting Point&lt;/h2&gt;

&lt;p&gt;Excursions has been a home for notes on cloud infrastructure, DevOps, and systems reliability since 2020. Over the years, the design stayed functional but hadn’t kept up with modern web aesthetics. The typography felt dated, the layout was utilitarian, and dark mode was an afterthought with inconsistent contrast.&lt;/p&gt;

&lt;p&gt;J decided it was time for a visual refresh — not a rewrite, but a thoughtful modernization of the existing Jekyll site. I (Claude) was brought in to pair on the implementation.&lt;/p&gt;

&lt;h2 id=&quot;what-changed&quot;&gt;What Changed&lt;/h2&gt;

&lt;h3 id=&quot;hero-section-and-navigation&quot;&gt;Hero Section and Navigation&lt;/h3&gt;

&lt;p&gt;The homepage got a new hero section with a concise tagline and quick-access links (About, GitHub, RSS). The navigation bar gained a glassmorphic effect — translucent with a backdrop blur that becomes solid on scroll. The font was swapped from IBM Plex Sans to &lt;strong&gt;Inter&lt;/strong&gt;, with tighter letter-spacing on headings and a refined weight scale.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/modernize/home-light.png&quot; alt=&quot;Homepage with hero section, popular cards, and latest articles&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;popular-posts-and-article-cards&quot;&gt;Popular Posts and Article Cards&lt;/h3&gt;

&lt;p&gt;A new &lt;strong&gt;Popular&lt;/strong&gt; section highlights three posts as gradient cards in a bento-style grid, selected based on actual site analytics. Article cards below received rounded corners, layered shadows, a subtle hover lift animation, and tag pills. Images now display at their natural aspect ratio — much better for technical screenshots.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/modernize/popular-cards.png&quot; alt=&quot;Popular section and article cards with scrolled navbar&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;sidebar-modernization&quot;&gt;Sidebar Modernization&lt;/h3&gt;

&lt;p&gt;The post page sidebar — showing Topics, Tags, and Related posts — was updated from a bordered box style to clean uppercase section labels with minimal dividers. Tags became pill buttons, and related posts are listed without bullets for a quieter visual.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/modernize/sidebar.png&quot; alt=&quot;Post page with modernized sidebar&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;simplified-footer&quot;&gt;Simplified Footer&lt;/h3&gt;

&lt;p&gt;The old three-column footer duplicated links already present in the header and hero section. It was replaced with a single row: copyright on the left, social links on the right. Clean and non-redundant.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/modernize/footer.png&quot; alt=&quot;Footer and pagination&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;dark-mode&quot;&gt;Dark Mode&lt;/h3&gt;

&lt;p&gt;Dark mode was polished from the ground up — deep charcoal background, high-contrast body text (WCAG AAA), lighter card surfaces with subtle borders, and Material Blue 200 links for comfortable reading.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/modernize/home-dark.png&quot; alt=&quot;Dark mode homepage&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;small-fixes&quot;&gt;Small Fixes&lt;/h3&gt;

&lt;ul&gt;
  &lt;li&gt;Search box text was invisible on the transparent navbar — fixed with state-aware input colors&lt;/li&gt;
  &lt;li&gt;Pagination got pill-style buttons matching the overall design language&lt;/li&gt;
  &lt;li&gt;Scroll-based animations (fade-in) were added for article cards&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;the-process&quot;&gt;The Process&lt;/h2&gt;

&lt;p&gt;The entire redesign was done in a single branch (&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;revamp&lt;/code&gt;) over a few pairing sessions. No build tools or preprocessors were added — it’s still plain CSS, Bootstrap 5.3.3, and Jekyll. Changes were verified live with a local dev server, iterating on feedback in real time.&lt;/p&gt;

&lt;p&gt;One recurring challenge was Jekyll’s dev server aggressively caching CSS files, which made it difficult to see updates. The workaround was moving some critical styles inline into HTML templates — a pragmatic solution that works reliably.&lt;/p&gt;

&lt;h2 id=&quot;wrapping-up&quot;&gt;Wrapping Up&lt;/h2&gt;

&lt;p&gt;The blog now has a modern, cohesive look while keeping its lightweight foundation. No JavaScript frameworks, no build pipelines, no complexity creep — just cleaner CSS and better visual hierarchy.&lt;/p&gt;

&lt;p&gt;This is the only post on Excursions written by an AI. Going forward, all content will be authored by J, as it has always been. Thanks for reading.&lt;/p&gt;

</description>
        <pubDate>Thu, 12 Mar 2026 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/blog/modernizing-excursions/</link>
        <guid isPermaLink="true">https://iam-j.github.io/blog/modernizing-excursions/</guid>
        
        <category>blog</category>
        
        <category>design</category>
        
        <category>ai</category>
        
        
        <category>blog</category>
        
      </item>
    
      <item>
        <title>Policy-as-code for Infrastructure</title>
        <description>&lt;p&gt;If your infrastructure has been deployed and maintained as code, chances are that you might be using Terraform or CloudFormation.In those cases most probably the enforcement of policies is after thought or takes backseat.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://www.checkov.io/&quot;&gt;Checkov&lt;/a&gt; is the tool with most common policies and checks, available as code to run and scan on the most Infrastructure as code files and return us misconfiguration or missing best practices.&lt;/p&gt;


&lt;p&gt;The steps from &lt;a href=&quot;https://www.checkov.io/1.Welcome/Quick%20Start.html#install-checkov-from-pypi&quot;&gt;Checkov Quick Start&lt;/a&gt; are straight forward and I see no point in repeating them here again.&lt;/p&gt;

&lt;p&gt;In additon there is a scope for expanding the functionality of the basic tool by defining your own personal/enterprise rules as &lt;a href=&quot;https://www.checkov.io/3.Custom%20Policies/Custom%20Policies%20Overview.html&quot;&gt;custom policies&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you insist on better visualisation or centralised location to track the IAC scan results, they have provided a &lt;a href=&quot;https://www.checkov.io/2.Basics/Visualizing%20Checkov%20Output.html&quot;&gt;basic web visualization&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Go ahead and give a try to see how compliant is your current Infrastructure as Code.&lt;/p&gt;

&lt;p&gt;Few screenshots from my run on the cloud formation scripts from  &lt;a href=&quot;https://github.com/aws-quickstart/quickstart-atlassian-jira&quot;&gt;here&lt;/a&gt;are below&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Running checkov&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/pac/pac.png&quot; alt=&quot;pac&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Sample Result from the run&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/pac/pac1.png&quot; alt=&quot;pac&quot; /&gt;&lt;/p&gt;

</description>
        <pubDate>Wed, 19 Jul 2023 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/iac/policy-as-code/</link>
        <guid isPermaLink="true">https://iam-j.github.io/iac/policy-as-code/</guid>
        
        <category>iac</category>
        
        <category>cloudformation</category>
        
        
        <category>iac</category>
        
      </item>
    
      <item>
        <title>Track CloudFormation Stack Changes Using EventBridge</title>
        <description>&lt;p&gt;If you are handling your infrastructure as code using CloudFormation stacks and is managed by multiple users, you could have run into a scenario where one of them might have run the stack or modified the stack and no one is aware of change. To track and get alert on those scenarios we now have support for CloudFormation &lt;a href=&quot;https://aws.amazon.com/about-aws/whats-new/2022/07/aws-cloudformation-event-notifications-amazon-eventbridge-event-driven-applications/&quot;&gt;events&lt;/a&gt; in the EventBridge which can be piped to SNS or custom alert system.&lt;/p&gt;

&lt;p&gt;Let us create a sample event to track Changes to one of the stacks.
Most of the steps are clearly described &lt;a href=&quot;https://aws.amazon.com/blogs/devops/using-cloudformation-events-to-build-custom-workflows-for-post-provisioning-management/&quot;&gt;here&lt;/a&gt; but the JSON here have few additional fields and statuses.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#json-for-our-custom-event&quot; id=&quot;markdown-toc-json-for-our-custom-event&quot;&gt;JSON for our custom event&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#eventbridge-rule&quot; id=&quot;markdown-toc-eventbridge-rule&quot;&gt;EventBridge Rule&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;json-for-our-custom-event&quot;&gt;JSON for our custom event&lt;/h3&gt;

&lt;p&gt;Below JSON will help us in creating an alert for different statuses like&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Creation Complete/Failed&lt;/li&gt;
  &lt;li&gt;Update Complete/Failed&lt;/li&gt;
  &lt;li&gt;Delete Failure&lt;/li&gt;
&lt;/ul&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-json&quot; data-lang=&quot;json&quot;&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
  &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;source&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;aws.cloudformation&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
  &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;detail-type&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;CloudFormation Resource Status Change&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
  &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;resources&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;arn:stack_Arn_here&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
  &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;detail&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
    &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;stack-id&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;arn:stack_Arn_here&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
    &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;status-details&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
      &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;status&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;CREATE_COMPLETE&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;CREATE_FAILED&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;UPDATE_FAILED&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;DELETE_FAILED&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;UPDATE_COMPLETE&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
    &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
  &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;h3 id=&quot;eventbridge-rule&quot;&gt;EventBridge Rule&lt;/h3&gt;

&lt;p&gt;Open AWS EventBridge and create a new rule with Custom Pattern as shown in the screenshot below&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/eventbridge/eventbridge1.jpg&quot; alt=&quot;eventbridge1&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Select your target which has to be SNS if you need to receive a notification&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/eventbridge/eventbridge2.jpg&quot; alt=&quot;eventbridge2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Once done create a rule and we should be good to receive the alert whenever a stack has been modified.&lt;/p&gt;

</description>
        <pubDate>Wed, 09 Nov 2022 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/iac/track-cloudformation-stack/</link>
        <guid isPermaLink="true">https://iam-j.github.io/iac/track-cloudformation-stack/</guid>
        
        <category>iac</category>
        
        <category>cloudformation</category>
        
        
        <category>iac</category>
        
      </item>
    
      <item>
        <title>Classic Load Balancer migration to ALB</title>
        <description>&lt;p&gt;AWS EC2 Classic have already been retired and the official &lt;a href=&quot;https://aws.amazon.com/blogs/aws/ec2-classic-is-retiring-heres-how-to-prepare/#:~:text=On%20October%2030%2C%202021%20we,Reserved%20Instances%20for%20EC2%2DClassic.&quot;&gt;deadline&lt;/a&gt; for migration is coming up.&lt;/p&gt;

&lt;p&gt;Although its not mentioned directly,the official recommendation is to use to the new Application/Network Load Balancers over the Classic Load Bakancers in VPC.&lt;/p&gt;

&lt;p&gt;We have a quick migration option available in the AWS management console but is not useful for Projects and Infra which are deployed/maintained using the Infrastructure as Code.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/clb/clb2.jpg&quot; alt=&quot;clb2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;In this post we will go through the steps necessary to perform the migration with zero downtime using Cloudformation templates on a sample application.&lt;/p&gt;

&lt;p&gt;Note:This sample application and cloud formation templates are internet facing and required further configuration and testing before using in live or prodcution environments.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#steps-inovlved-in-the-migration&quot; id=&quot;markdown-toc-steps-inovlved-in-the-migration&quot;&gt;Steps inovlved in the migration&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#prepare-sample-application&quot; id=&quot;markdown-toc-prepare-sample-application&quot;&gt;Prepare Sample application&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#deploy-alb-with-out-deletion-of-the-classic-load-balancer&quot; id=&quot;markdown-toc-deploy-alb-with-out-deletion-of-the-classic-load-balancer&quot;&gt;Deploy ALB with out deletion of the Classic Load balancer&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#clean-old-classic-load-balancer&quot; id=&quot;markdown-toc-clean-old-classic-load-balancer&quot;&gt;Clean Old Classic Load Balancer&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#conclusion&quot; id=&quot;markdown-toc-conclusion&quot;&gt;Conclusion&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;steps-inovlved-in-the-migration&quot;&gt;Steps inovlved in the migration&lt;/h3&gt;

&lt;ul&gt;
  &lt;li&gt;Check and validate your existing Cloudformation template against exsiting Infrastructure.&lt;/li&gt;
  &lt;li&gt;Update the template with Application Load balancer changes.&lt;/li&gt;
  &lt;li&gt;Create new ALB while still serving traffic through the old Classic load balancer.&lt;/li&gt;
  &lt;li&gt;Slowly divert traffic to the new ALB.&lt;/li&gt;
  &lt;li&gt;Delete old classic load balancer using the Cloud formation.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I have executed above steps with a sample application and explained below&lt;/p&gt;

&lt;h3 id=&quot;prepare-sample-application&quot;&gt;Prepare Sample application&lt;/h3&gt;

&lt;p&gt;The Cloudformation template &lt;a href=&quot;https://gist.github.com/iam-j/9f20795093061aedb1763f692e34be5a&quot;&gt;here&lt;/a&gt; will create a Classic load balancer pointing to our sample application hosted on EC2.&lt;/p&gt;

&lt;p&gt;The output of the same should be as below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/clb/clb3.jpg&quot; alt=&quot;clb3&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;deploy-alb-with-out-deletion-of-the-classic-load-balancer&quot;&gt;Deploy ALB with out deletion of the Classic Load balancer&lt;/h3&gt;

&lt;p&gt;This CF &lt;a href=&quot;https://gist.github.com/iam-j/b3876fe8f5f34673db5a6ce42d21cb32&quot;&gt;template&lt;/a&gt; will create a new Application load balancer which is going to replace our Classic load balancer created in step one.This is to ensure our new URL is ready and serving traffic before deleting the old endpoint.&lt;/p&gt;

&lt;p&gt;The new URL should be serving our app.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/clb/clb4.jpg&quot; alt=&quot;clb4&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;clean-old-classic-load-balancer&quot;&gt;Clean Old Classic Load Balancer&lt;/h3&gt;

&lt;p&gt;Once we are sure that the traffic is moved to the new URL i.e Application Load Balancer,we are good to run the last &lt;a href=&quot;https://gist.github.com/iam-j/ea936dcdd5922b76e2cd720895899a08&quot;&gt;template&lt;/a&gt; which would delete our Classic Load Balancer.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/clb/clb5.jpg&quot; alt=&quot;clb5&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;

&lt;p&gt;Using this 3 step process we could easily migrate off from the Classic load balancer to Applcation load balancer with zero down time, which is very much need and appreciated on production environments.&lt;/p&gt;

</description>
        <pubDate>Sat, 16 Apr 2022 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/aws/clb-migration-alb/</link>
        <guid isPermaLink="true">https://iam-j.github.io/aws/clb-migration-alb/</guid>
        
        <category>aws</category>
        
        <category>iac</category>
        
        
        <category>aws</category>
        
      </item>
    
      <item>
        <title>Simple Ansible playbook to get the Linux host uptime</title>
        <description>&lt;p&gt;While working with EC2 Linux instances there could be a requirement to get the list of all the machines which had not been rebooted over months or for a particular number of days.The reason could be for audit or to the check the status of instances.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#assumptions&quot; id=&quot;markdown-toc-assumptions&quot;&gt;Assumptions&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#script&quot; id=&quot;markdown-toc-script&quot;&gt;Script&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#whats-next&quot; id=&quot;markdown-toc-whats-next&quot;&gt;What’s next&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I have put together tasks to accomplish it as an Ansible Playbook below.&lt;/p&gt;

&lt;h3 id=&quot;assumptions&quot;&gt;Assumptions&lt;/h3&gt;

&lt;p&gt;You have an working Ansible environment with a proper inventory file with all the details of hosts being managed.&lt;/p&gt;

&lt;h3 id=&quot;script&quot;&gt;Script&lt;/h3&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-yaml&quot; data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;pi&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;na&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;Get the list of all the nodes which are running over a month&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;hosts&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;all&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;gather_facts&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;false&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;tasks&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;

      &lt;span class=&quot;pi&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;na&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;Show hostname for sanity check (or just use -v flag for verbose)&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;command&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;hostname&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;register&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;hostname&lt;/span&gt;
      &lt;span class=&quot;pi&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;na&quot;&gt;debug&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;var=hostname.stdout_lines&lt;/span&gt;

      &lt;span class=&quot;pi&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;na&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;Check uptime prior reboot&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;shell&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;cut -d &apos; &apos; -f1 /proc/uptime&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;register&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;UPTIME_PRE_REBOOT&lt;/span&gt;

      &lt;span class=&quot;pi&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;na&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;Setting fact for number of days&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;set_fact&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;
          &lt;span class=&quot;na&quot;&gt;uptime_days&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;(UPTIME_PRE_REBOOT.stdout&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;int&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;/&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;86400)&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;round(0)&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;}&quot;&lt;/span&gt;

      &lt;span class=&quot;pi&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;na&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;Hosts to be rebooted.&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;debug&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;
          &lt;span class=&quot;na&quot;&gt;msg&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;has&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;not&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;been&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;rebooted&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;since&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;  &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;days,which&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;is&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;older&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;than&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;a&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;month&quot;&lt;/span&gt;
        &lt;span class=&quot;na&quot;&gt;when&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;(uptime_days | int) &amp;gt; 30&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;This simple playbook fetches the uptime and converts to days,which can be used as condition.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/uptime1.jpg&quot; alt=&quot;uptime1&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;whats-next&quot;&gt;What’s next&lt;/h3&gt;

&lt;p&gt;This script can be extended to take an action on the hosts which have not been rebooted.&lt;/p&gt;

</description>
        <pubDate>Wed, 02 Feb 2022 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/iac/ansible-machine-uptime/</link>
        <guid isPermaLink="true">https://iam-j.github.io/iac/ansible-machine-uptime/</guid>
        
        <category>iac</category>
        
        <category>ansible</category>
        
        
        <category>iac</category>
        
      </item>
    
      <item>
        <title>Mitigating file system failures on root EBS volume</title>
        <description>&lt;p&gt;Below scenario explains the troubleshooting steps to get the EC2 recovered from EBS boot failiures.&lt;/p&gt;

&lt;p&gt;Issue: System root has failed consistency check and instance was no longer able to boot.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/ebs/ebs1.jpg&quot; alt=&quot;ebs1&quot; /&gt;&lt;/p&gt;


&lt;p&gt;Procedure to be followed for bringing the EC2 instance back.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Capture the current EBS volume metadata from AWS console,especially the device mount name.
&lt;img src=&quot;https://iam-j.github.io/static/img/ebs/ebs2.jpg&quot; alt=&quot;ebs2&quot; /&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Stop the EC2 instance which needs to be troubleshooted.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Detach the volume which has to be fixed from the AWS management console.
&lt;img src=&quot;https://iam-j.github.io/static/img/ebs/ebs3.jpg&quot; alt=&quot;ebs3&quot; /&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Use the same console to attach the volume to the rescue machine as a secondary device &lt;strong&gt;(/dev/sdf)&lt;/strong&gt;.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Check if the disk available on the rescue EC2 instance using&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;    lsblk
    &lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;ul&gt;
  &lt;li&gt;Run the &lt;em&gt;fsck&lt;/em&gt; on the drive to fix the errors.&lt;/li&gt;
&lt;/ul&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;    fsck &amp;lt;mount&amp;gt;
    &lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Ex: fsck /dev/nvme2n1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;ul&gt;
  &lt;li&gt;If you see an error with the above command, run fsck directly.&lt;/li&gt;
&lt;/ul&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;    fsck
    &lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Above commands can be run with -N flag as a dry run to see the changes to be made.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Once the file errors are fixed,detach the EBS volume from the rescue EC2 instance and attach to the instance which has been taken down for maintenance.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Start the EC2 instance and check the system log.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;If the process is successful we should be able to boot into the instance and the services would be running again.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This &lt;a href=&quot;https://aws.amazon.com/premiumsupport/knowledge-center/ec2-linux-status-check-failure-os-errors/&quot;&gt;url&lt;/a&gt; has additional troubleshooting for EBS volumes.&lt;/p&gt;

</description>
        <pubDate>Mon, 20 Dec 2021 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/aws/ec2-disk-corruption/</link>
        <guid isPermaLink="true">https://iam-j.github.io/aws/ec2-disk-corruption/</guid>
        
        <category>aws</category>
        
        <category>ec2</category>
        
        <category>ebs</category>
        
        
        <category>aws</category>
        
      </item>
    
      <item>
        <title>Python sample script to fetch data from Jenkins API</title>
        <description>&lt;p&gt;While working with Jenkins,there was a requirement to collect details of the Jobs/builds that had been run on the Server and derive statistics like&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Total Runs of a job:&lt;/li&gt;
  &lt;li&gt;Total Success:&lt;/li&gt;
  &lt;li&gt;Total Failures and etc.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/jenapi/jenapi2.jpg&quot; alt=&quot;jenapi2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;There could be different approaches to capture the information and the UI also shows the few trends directly,but I took the help of Python and Jenkins API to get it done.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#jenkins-api&quot; id=&quot;markdown-toc-jenkins-api&quot;&gt;Jenkins API&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#access-token&quot; id=&quot;markdown-toc-access-token&quot;&gt;Access token&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#script&quot; id=&quot;markdown-toc-script&quot;&gt;Script&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#whats-next&quot; id=&quot;markdown-toc-whats-next&quot;&gt;What’s next&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;jenkins-api&quot;&gt;Jenkins API&lt;/h3&gt;

&lt;p&gt;Jenkins provides REST Api which allows us to perform most of the actions and fetch the information which is usually available in the UI, programmatically.We can use the official Jenkins Builds &lt;a href=&quot;https://ci.jenkins.io/&quot;&gt;URL&lt;/a&gt; and its corresponding &lt;a href=&quot;https://ci.jenkins.io/api/&quot;&gt;api&lt;/a&gt; as an example to go through the options available.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Do remember the official Jenkins server has restricted access to view build and job details from api.You should have full information available on your self hosted server though as shown below&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/jenapi/jenapi3.jpg&quot; alt=&quot;jenapi3&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;access-token&quot;&gt;Access token&lt;/h3&gt;

&lt;p&gt;Although the …./API/json details can be seen on the browser,you actually need configure an access token to capture them programmatically.So please go ahead and generate it using the steps already detailed on official &lt;a href=&quot;https://support.cloudbees.com/hc/en-us/articles/115003090592-How-to-re-generate-my-Jenkins-user-token&quot;&gt;guide&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;script&quot;&gt;Script&lt;/h3&gt;

&lt;p&gt;Now lets come to the actual script which has been written in haste to complete the task, but it should be an useful snippet for a project to build up on.&lt;/p&gt;

&lt;p&gt;For every URL we see in the Jenkins UI, we can append &lt;strong&gt;/api/json&lt;/strong&gt; to see its JSON equivalent.In this script,we are fetching JSON data from the URL’s of the job and its executed builds.&lt;/p&gt;

&lt;p&gt;First,we get the latest build ran and go through all its runs to get the statuses.So the complete script follows&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;requests&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;json&lt;/span&gt;

&lt;span class=&quot;c1&quot;&gt;#Your jenkins URL and credentials goes here
&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;url&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&apos;https://jenkins-server.com/job/folder/job/Jobname/api/json&apos;&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;username&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&apos;username&apos;&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;password&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&apos;you access token&apos;&lt;/span&gt;


&lt;span class=&quot;c1&quot;&gt;#Use the &apos;auth&apos; parameter to send requests with HTTP Basic Auth:
#Accessing the Job page to get the latest Build ran.
&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;response&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;post&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;url&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;auth&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;username&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;password&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;verify&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;False&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;try&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;buildnumberJson&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;json&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;loads&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;response&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;except&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Failed to parse json&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;lastBuild&quot;&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;buildnumberJson&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;  
    &lt;span class=&quot;n&quot;&gt;totalbuilds&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;buildnumberJson&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;lastBuild&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;runs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;totalbuilds&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;number&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
    
&lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Failed to get build&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;c1&quot;&gt;#Iterate over the job build runs to get the build status for each
&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;totalsuccess&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;totalfailure&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;totalmissing&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;build&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;range&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;runs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;buildurl&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&apos;https://jenkins-server.com/job/folder/job/Jobname/api/json&apos;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;str&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;build&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&apos;/api/json&apos;&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;buildurl&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;buildstatus&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[]&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;try&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;response&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;requests&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;post&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;buildurl&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;auth&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;username&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;password&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;verify&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;False&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;buildstatus&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;json&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;loads&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;response&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;except&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;Exception&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;as&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;e&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;totalmissing&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;  &lt;span class=&quot;n&quot;&gt;totalmissing&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;  
    &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;result&quot;&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;buildstatus&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;buildstatus&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;result&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;SUCCESS&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
            &lt;span class=&quot;n&quot;&gt;totalsuccess&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;totalsuccess&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;buildstatus&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;result&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;FAILURE&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;
            &lt;span class=&quot;n&quot;&gt;totalfailure&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;totalfailure&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;


&lt;span class=&quot;c1&quot;&gt;#Generate Output numbers
&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;sa&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;total builds:&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;runs&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;sa&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;total succeeded builds:&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;totalsuccess&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;sa&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;total failed builds:&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;totalfailure&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;sa&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;total skipped builds:&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;totalmissing&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;h3 id=&quot;whats-next&quot;&gt;What’s next&lt;/h3&gt;

&lt;p&gt;This script can be extended to get details of the jobs on &lt;em&gt;server&lt;/em&gt; and their success,time to run etc and build custom stats which are not available by default in Jenkins.Also we can start managing jobs programmatically with out manually logging into the console.&lt;/p&gt;

</description>
        <pubDate>Sun, 28 Mar 2021 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/jenkins/python-sample-script-to-jenkins/</link>
        <guid isPermaLink="true">https://iam-j.github.io/jenkins/python-sample-script-to-jenkins/</guid>
        
        <category>python</category>
        
        <category>jenkins</category>
        
        
        <category>jenkins</category>
        
      </item>
    
      <item>
        <title>Voila:Host your Jupyter Notebook as webpage</title>
        <description>&lt;p&gt;&lt;strong&gt;Scenario:&lt;/strong&gt; As a continuation from previous post &lt;a href=&quot;https://iam-j.github.io/s3/s3-browse-download-no-publicaccess/https://iam-j.github.io/s3/s3-browse-download-no-publicaccess/&quot;&gt;here&lt;/a&gt; where users have to access a Jupyter Notebook to browse the files,we now have to create a website/webpage to have seamless experience hiding the code and notebook interface.&lt;/p&gt;

&lt;p&gt;To achieve this we are going to use &lt;a href=&quot;https://github.com/voila-dashboards/voila/tree/stable&quot;&gt;Voila&lt;/a&gt;, an opensource tool/extension.&lt;/p&gt;

&lt;p&gt;Lets see how to build web server,serving the notebook.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#preparing-ec2-to-host-jupyter&quot; id=&quot;markdown-toc-preparing-ec2-to-host-jupyter&quot;&gt;Preparing EC2 to host Jupyter&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#install-voila&quot; id=&quot;markdown-toc-install-voila&quot;&gt;Install Voila&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#update-notebook&quot; id=&quot;markdown-toc-update-notebook&quot;&gt;Update Notebook&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#launch-voila&quot; id=&quot;markdown-toc-launch-voila&quot;&gt;Launch Voila&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#notebook-as-webpage&quot; id=&quot;markdown-toc-notebook-as-webpage&quot;&gt;Notebook as Webpage&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#conclusion&quot; id=&quot;markdown-toc-conclusion&quot;&gt;Conclusion&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;preparing-ec2-to-host-jupyter&quot;&gt;Preparing EC2 to host Jupyter&lt;/h3&gt;

&lt;p&gt;We are going repeat the same steps in previous to have an EC2 with Jupyter prerequisites installed.&lt;/p&gt;

&lt;p&gt;Create an EC2 to serve as a Bastion host with&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;Jupyter Notebook&lt;/li&gt;
  &lt;li&gt;AWS CLI&lt;/li&gt;
  &lt;li&gt;Python Boto3&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This can be achieved by steps mentioned &lt;a href=&quot;https://dataschool.com/data-modeling-101/running-jupyter-notebook-on-an-ec2-server/&quot;&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;install-voila&quot;&gt;Install Voila&lt;/h3&gt;

&lt;p&gt;We will run voila server standalone with out using the extension,so proceed directly installing from &lt;strong&gt;Pip&lt;/strong&gt;&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;pip3 &lt;span class=&quot;nb&quot;&gt;install &lt;/span&gt;voila&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;h3 id=&quot;update-notebook&quot;&gt;Update Notebook&lt;/h3&gt;

&lt;p&gt;Update the earlier notebook with the below Python code to access the S3 files.This code will prompt user for the bucket name and lists all the files available.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-python&quot; data-lang=&quot;python&quot;&gt;&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;ipywidgets&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;widgets&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;boto3&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;lbl1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;widgets&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Label&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Enter Bucket Name&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;display&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;lbl1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;text1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;widgets&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;display&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;btn&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;widgets&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Button&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;description&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;List&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;out&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;widgets&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Output&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;list_files&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;_&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;    
    
    &lt;span class=&quot;n&quot;&gt;s3&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;boto3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;resource&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&apos;s3&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;my_bucket&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;s3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Bucket&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;str&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;text1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;value&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
   
    &lt;span class=&quot;k&quot;&gt;with&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;out&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;        
        &lt;span class=&quot;n&quot;&gt;out&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;clear_output&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;my_bucket_object&lt;/span&gt; &lt;span class=&quot;ow&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;my_bucket&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;objects&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;all&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;():&lt;/span&gt;
            &lt;span class=&quot;k&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;my_bucket_object&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;key&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        

    
&lt;span class=&quot;n&quot;&gt;btn&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;on_click&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;list_files&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;widgets&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;VBox&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;([&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;btn&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;out&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/voila/voila2.jpg&quot; alt=&quot;voila2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Note that the code can be extended to show all the buckets and then asking the user to select the one he needs.&lt;/p&gt;

&lt;h3 id=&quot;launch-voila&quot;&gt;Launch Voila&lt;/h3&gt;

&lt;p&gt;Next, navigate to the folder where the notebook exists and launch voila.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-bash&quot; data-lang=&quot;bash&quot;&gt;voila&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;This will start a webserver hosting all the notebooks available.We can even add a Ngnix server in front of this service to serve multiple requests.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/voila/voila4.jpg&quot; alt=&quot;voila4&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;notebook-as-webpage&quot;&gt;Notebook as Webpage&lt;/h3&gt;

&lt;p&gt;Once the link to notebook has been clicked,you would notice that Voila would run all the cells and gives an output as webpage.In our case it displays a text box and prompts for user input for &lt;em&gt;bucket name&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/voila/voila1.jpg&quot; alt=&quot;voila1&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Enter the bucket name and click &lt;em&gt;List&lt;/em&gt;,you should have the required output&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/voila/voila3.jpg&quot; alt=&quot;voila3&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Please note that the burden to show the UI and widgets falls on us to ease the look and experience.In this example, i have used simple &lt;em&gt;Ipywidgets&lt;/em&gt; to get the job done.we have a lot of complex available in the official Voila gallery &lt;a href=&quot;https://voila-gallery.org/&quot;&gt;here&lt;/a&gt; with their source code.&lt;/p&gt;

&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;

&lt;p&gt;So, Using this we can make and serve simple web apps using Jupyter notebooks.This will be very useful when the users are not comfortable enough to run notebooks and we have limited time to create a webapp.&lt;/p&gt;

</description>
        <pubDate>Thu, 21 Jan 2021 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/jupyter/voila-jupyter-as-webpage/</link>
        <guid isPermaLink="true">https://iam-j.github.io/jupyter/voila-jupyter-as-webpage/</guid>
        
        <category>aws</category>
        
        <category>ec2</category>
        
        <category>s3</category>
        
        <category>jupyter</category>
        
        
        <category>jupyter</category>
        
      </item>
    
      <item>
        <title>Browse and download S3 files with out public access to Bucket</title>
        <description>&lt;p&gt;&lt;strong&gt;Scenario:&lt;/strong&gt; Users have to access and download files from a S3 bucket but not upload or change the contents of the same.&lt;/p&gt;

&lt;p&gt;We can address the requirement by following official documented steps &lt;a href=&quot;https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteAccessPermissionsReqd.html&quot;&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;To make your bucket publicly readable, you must disable block public access settings for the bucket and write a bucket policy that grants public read access. If your bucket contains objects that are not owned by the bucket owner, you might also need to add an object access control list (ACL) that grants everyone read access.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;But for my use case the bucket cannot be made public, users should be able to browse the files in S3 as good as any person with AWS login credentials with CLI or Python while retaining the read only access.&lt;/p&gt;

&lt;p&gt;Hence, Jupyter Notebooks comes to my rescue to build a solution as described in this post.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#preparing-ec2-to-host-jupyter&quot; id=&quot;markdown-toc-preparing-ec2-to-host-jupyter&quot;&gt;Preparing EC2 to host Jupyter&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#create-s3-read-only-instance-role&quot; id=&quot;markdown-toc-create-s3-read-only-instance-role&quot;&gt;Create S3 read only instance role&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#create-default-notebook&quot; id=&quot;markdown-toc-create-default-notebook&quot;&gt;Create default Notebook&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#download-files&quot; id=&quot;markdown-toc-download-files&quot;&gt;Download Files&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#conclusion&quot; id=&quot;markdown-toc-conclusion&quot;&gt;Conclusion&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;preparing-ec2-to-host-jupyter&quot;&gt;Preparing EC2 to host Jupyter&lt;/h3&gt;

&lt;p&gt;Create an EC2 to serve as a Bastion host with&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;Jupyter Notebook&lt;/li&gt;
  &lt;li&gt;AWS CLI&lt;/li&gt;
  &lt;li&gt;Python Boto3&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This can be achieved by steps mentioned &lt;a href=&quot;https://dataschool.com/data-modeling-101/running-jupyter-notebook-on-an-ec2-server/&quot;&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;At the end, we have a Jupyter notebook &lt;strong&gt;URL&lt;/strong&gt; generated which users can access from any browser.&lt;/p&gt;

&lt;h3 id=&quot;create-s3-read-only-instance-role&quot;&gt;Create S3 read only instance role&lt;/h3&gt;

&lt;p&gt;Next create EC2 instance role with read only access policy for the S3 as below.&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-json&quot; data-lang=&quot;json&quot;&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
    &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;Version&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;2012-10-17&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
    &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;Statement&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
        &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
            &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;Effect&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;Allow&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
            &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;Action&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
                &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;s3:Get*&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
                &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;s3:List*&quot;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
            &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
            &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;Resource&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;arn:aws:s3:::my_bucket&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
                 &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;arn:aws:s3:::my_bucket/*&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
        &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
    &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;p&gt;Attach this role to the EC2 and we are good to access all the S3 files from it.But wait we are not going to give EC2 access directly to the user.&lt;/p&gt;

&lt;h3 id=&quot;create-default-notebook&quot;&gt;Create default Notebook&lt;/h3&gt;

&lt;p&gt;Jupyter URL generated in the first step now act as a webservice running on our Bastion Ec2 and allows required S3 access to the users.&lt;/p&gt;

&lt;p&gt;Now, to make the work easy for non-technical user.We can create a notebook with all the required S3 commands and save it in the default directory&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/s3note/s3note1.jpg&quot; alt=&quot;s3note1&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The notebook has Commands to&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;Browse files in S3.&lt;/li&gt;
  &lt;li&gt;Copy files from S3 to temp directory on EC2 machine.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;download-files&quot;&gt;Download Files&lt;/h3&gt;

&lt;p&gt;Once we have followed all the steps, we should see our required file from the S3 available to download in temp directory.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/s3note/s3note2.jpg&quot; alt=&quot;s3note2&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Select the file, click download and we are good to use the file.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/s3note/s3note3.jpg&quot; alt=&quot;s3note3&quot; /&gt;&lt;/p&gt;

&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;

&lt;p&gt;There might be better ways to achieve the same result and this could have downside of user misusing the EC2 machine.But at the end,it is quick and easiest solution I could put together and get working.&lt;/p&gt;

&lt;p&gt;Also, Jupyter Hub can be used instead of notebook for more granular control and scaling for multiple users.&lt;/p&gt;

</description>
        <pubDate>Mon, 07 Dec 2020 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/s3/s3-browse-download-no-publicaccess/</link>
        <guid isPermaLink="true">https://iam-j.github.io/s3/s3-browse-download-no-publicaccess/</guid>
        
        <category>aws</category>
        
        <category>ec2</category>
        
        <category>s3</category>
        
        
        <category>s3</category>
        
      </item>
    
      <item>
        <title>Managing EC2 AMI used in ECS clusters</title>
        <description>&lt;p&gt;People using EC2 backed ECS cluster can understand the pain which goes in to maintaining and patching the base machines running our ECS containers.We usually pick two paths&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Use ECS optimized AMI maintained by AWS.&lt;/li&gt;
  &lt;li&gt;Use custom AMI with patching on our convenience.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Let us the see pros and cons with both the methods.&lt;/p&gt;

&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#ecs-optimized-ami-images&quot; id=&quot;markdown-toc-ecs-optimized-ami-images&quot;&gt;ECS optimized AMI images&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#self-managed-ecs-ami&quot; id=&quot;markdown-toc-self-managed-ecs-ami&quot;&gt;Self managed ECS AMI&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#finally&quot; id=&quot;markdown-toc-finally&quot;&gt;Finally&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;ecs-optimized-ami-images&quot;&gt;ECS optimized AMI images&lt;/h3&gt;

&lt;p&gt;&lt;img src=&quot;https://iam-j.github.io/static/img/awsami1.jpg&quot; alt=&quot;awsami1&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Pros:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;AWS updates base AMI periodically for us with latest security patches.&lt;/li&gt;
  &lt;li&gt;AWS regularly pushes AMI with updated ECS agent,Docker versions and dependencies.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Cons:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Older AMI’s disappears and will not available.&lt;/li&gt;
  &lt;li&gt;If you have hardcoded your Launch template/Configuration,cluster goes down when AMI is no longer available.&lt;/li&gt;
  &lt;li&gt;Compatibility issues may occur with the newer versions,especially in case of Windows Containers.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Solution to prevent hard coded AMI names using Terraform.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;We can create Terraform resource as below to always fetch latest ECS optimized image provided by Amazon and then use it in our launch configuration or template&lt;/p&gt;

&lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-hcl&quot; data-lang=&quot;hcl&quot;&gt;&lt;span class=&quot;nx&quot;&gt;data&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;aws_ami&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;windows&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;most_recent&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;owners&lt;/span&gt;      &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;amazon&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;filter&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;nx&quot;&gt;name&lt;/span&gt;   &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;name&quot;&lt;/span&gt;
    &lt;span class=&quot;nx&quot;&gt;values&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;Windows_Server-2019-English-Full-ECS_Optimized-*&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;nx&quot;&gt;resource&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;aws_launch_configuration&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;windows_ecs&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;associate_public_ip_address&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;true&quot;&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;enable_monitoring&lt;/span&gt;           &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;true&quot;&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;iam_instance_profile&lt;/span&gt;        &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;arn:aws:iam::XXXXXXXX&quot;&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;image_id&lt;/span&gt;                    &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;data&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;aws_ami&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;windows&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;id&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;instance_type&lt;/span&gt;               &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;t3.xlarge&quot;&lt;/span&gt;
  &lt;span class=&quot;nx&quot;&gt;key_name&lt;/span&gt;                    &lt;span class=&quot;p&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;my key
  name_prefix                 = &quot;&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;ecs&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;windows&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;
  root_block_device {
    delete_on_termination = &quot;&lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;
    encrypted             = &quot;&lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;false&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;
    iops                  = &quot;&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;
    volume_size           = &quot;&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;50&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;
    volume_type           = &quot;&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;gp2&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;
  }

  lifecycle { create_before_destroy = true }
}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;

&lt;h3 id=&quot;self-managed-ecs-ami&quot;&gt;Self managed ECS AMI&lt;/h3&gt;

&lt;p&gt;Pros:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;We can retain total control of ownership and maintenance of the AMI image.&lt;/li&gt;
  &lt;li&gt;We can enforce additional security and custom patching cycles.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Cons:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Obviously the burden to maintain the images falls on us.&lt;/li&gt;
  &lt;li&gt;AMI storage cost might increase depending on our versioning strategy.&lt;/li&gt;
  &lt;li&gt;Resources are required to pick latest ECS agent, docker version etc., test and update.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;finally&quot;&gt;Finally&lt;/h3&gt;

&lt;p&gt;Whichever path we pick , the burden to change and upgrade our ECS cluster launch configuration falls on us.Although using and IAC like Terraform would make it easy to quickly update.&lt;/p&gt;

&lt;p&gt;Do let me know if there are bette ways to maintain the AMI and EC2 backed ECS clusters.&lt;/p&gt;

</description>
        <pubDate>Sun, 08 Nov 2020 00:00:00 +0000</pubDate>
        <link>https://iam-j.github.io/ecs/ec2-cluster-ecs-ami-manage/</link>
        <guid isPermaLink="true">https://iam-j.github.io/ecs/ec2-cluster-ecs-ami-manage/</guid>
        
        <category>aws</category>
        
        <category>ec2</category>
        
        <category>autoscale</category>
        
        
        <category>ecs</category>
        
      </item>
    
  </channel>
</rss>
