{"id":36,"date":"2023-12-08T11:50:08","date_gmt":"2023-12-08T11:50:08","guid":{"rendered":"https:\/\/blogs.imperial.ac.uk\/david-tang\/?p=36"},"modified":"2023-12-08T15:11:22","modified_gmt":"2023-12-08T15:11:22","slug":"reflect-programming-year","status":"publish","type":"post","link":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/","title":{"rendered":"Reflecting on programming in 2023"},"content":{"rendered":"<p>The year is coming to an end, and this is a good juncture to look back at 2023. This is a short rewind in 2023 from a programming and coding perspective.<\/p>\n<p>I had the opportunity to incorporate a couple of new frameworks in my work this year.<\/p>\n<p>Read below on my what I have experienced, and my takeaways.<\/p>\n<h2>1. Ploomber<\/h2>\n<p>This is a <a href=\"https:\/\/ploomber.io\/\">project management tool for a data science workflow<\/a>.<\/p>\n<p>It is similar to other frameworks such as Apache Airflow. It allows the project to be streamlined as a DAG (direct acyclic graph).<\/p>\n<figure style=\"width: 470px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/ploomber.io\/\"><img decoding=\"async\" src=\"https:\/\/ploomber.io\/images\/logo.png\" alt=\"Ploomber icon\" \/><\/a><figcaption class=\"wp-caption-text\"><em>Ploomber logo. Using an otter as a mascot is an automatic win for me.<\/em><\/figcaption><\/figure>\n<p>According to the developers, it is more suited than Airflow for ML projects. It is more targeted towards the development or explorative stage. What it does not aim to do is data orchestration and running regular production tasks.<\/p>\n<p><strong>Advantages of Ploomber:<\/strong><\/p>\n<p>&nbsp;<\/p>\n<ul>\n<li>low technical burden (just a package that can be installed in conda)<\/li>\n<li>can include unit tests, debugging<\/li>\n<li>keeps track of any new changes in project files<\/li>\n<li>easily rerun parts of the projects, where scripts have been edited<\/li>\n<li>can incorporate various languages such as SQL and, R within the pipeline<\/li>\n<\/ul>\n<p><strong>Disadvantages of Ploomber:<\/strong><\/p>\n<ul>\n<li>probably not needed for scripting and quick prototyping<\/li>\n<\/ul>\n<p><strong>Takeaway<\/strong>: Data projects are quickly evolving, and we need to be able to pivot flexibly at times. It is useful to have an equally flexible pipeline in place to take care of making sure upstream and downstream scripts &amp; outputs are kept up-to-date.<\/p>\n<h2>2. WSL2 and Bash scripting<\/h2>\n<p>This was more of a lesson for me in managing environments in a virtual machine.<\/p>\n<p>WSL2 = Windows Subsystem for Linux2. It is basically adding a<a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/wsl\/about\"> virtual Linux-based command line system that you can access from within Windows itself<\/a>.<\/p>\n<p>I am currently running Ubuntu with Bash on WSL2. Currently, I find it sufficient enough to do everything data science related from the command line. Git works nicely within WSL2 as well. And I can integrate everything via my favourite IDE (integrated development environment), Visual Studio Code.<\/p>\n<figure style=\"width: 230px\" class=\"wp-caption alignright\"><a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/wsl\/install\"><img decoding=\"async\" src=\"https:\/\/bendyworks.com\/assets\/images\/blog\/2020-05-21-setup-windows-development-eb5c79a7.png\" alt=\"WSL2\" \/><\/a><figcaption class=\"wp-caption-text\"><em>WSL2 &#8211; Best of both worlds, Unix and Windows<\/em><\/figcaption><\/figure>\n<p><strong>Pros:<\/strong><\/p>\n<ul>\n<li>can run pretty much anything, as if within a native Linux environment<\/li>\n<li>don&#8217;t need to fiddle with installing similar commands into the Windows command line (cmd)<\/li>\n<li>WSL2 is a one-liner setup on the Windows command line now, with newer versions of Windows<\/li>\n<\/ul>\n<p><strong>Cons:<\/strong><\/p>\n<ul>\n<li>sometimes the virtualisation gets a bit abstract<\/li>\n<li>need to be mindful to connect\/execute within the right command-line (WSL2 vs windows native)<\/li>\n<\/ul>\n<p><strong>Takeaway<\/strong>: With big data\/cloud computing, we&#8217;ll need to deal with remote machines at times. It can get quite abstract but beats running everything on a local machine at times. Getting comfortable with navigating Unix systems is handy.<\/p>\n<h2>3. Handlebars<\/h2>\n<p>This is a framework that was a bit of a tryout for me. It is based in the web development space, and not so much typical analytics programming.<\/p>\n<p>What is this? <a href=\"https:\/\/handlebarsjs.com\/\">It&#8217;s a template compiler<\/a>. Basically, it compiles into HTML and Javascript. Apparently, some code and speed optimisations happen under the hood as well.<\/p>\n<figure style=\"width: 254px\" class=\"wp-caption alignright\"><a href=\"https:\/\/handlebarsjs.com\/\"><img decoding=\"async\" class=\"\" src=\"https:\/\/handlebarsjs.com\/images\/handlebars_logo.png\" alt=\"handlebars icon\" \/><\/a><figcaption class=\"wp-caption-text\">Handlebars logo. Kind of reminds me of Mr. Potato Head for some reason.<\/figcaption><\/figure>\n<p>I had quite a bit of fun with using &#8216;<em>moustaches<\/em>&#8216; in my code too!<\/p>\n<p>If you don&#8217;t know what they are, they are these {{curly braces}}.\u00a0They are pretty versatile and can do a little bit more than your usual declared variable. For example, one can either load whole templates of website sections or declare helper functions with moustaches.<\/p>\n<p>Don&#8217;t have too much by ways of pros &amp; cons on this one, I&#8217;ll probably need a bit more time to explore it.<\/p>\n<p>With frontend web development, there is an abundance of frameworks (Can be a bit overwhelming!). It feels like I&#8217;m spoilt for choice to decide between a variety of frameworks just for Javascript alone (ie. Vue, Angular, React, etc).<\/p>\n<p><strong>Takeaway<\/strong>: Programming is just a tool and a means to an end. I see the frontend as a means to communicate and put ideas out there. I am pretty sure it will come in handy at some point!<\/p>\n<h2>Closing<\/h2>\n<p>I am really thankful to be able to geek out on these frameworks. It is like getting a new tool, testing it out, and actually building something from it.<\/p>\n<p>Perhaps, at this point, just looking at a few frameworks might not have amounted to something tangible in the real world. I&#8217;m sure it will all add up at a later time.<\/p>\n<blockquote><p><em>\u201cThe best way to predict the future is to create it\u201d -Peter Drucker <\/em><\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>The year is coming to an end, and this is a good juncture to look back at 2023. This is a short rewind in 2023 from a programming and coding perspective. I had the opportunity to incorporate a couple of new frameworks in my work this year. Read below on my what I have experienced, [&hellip;]<\/p>\n","protected":false},"author":1774,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,3,2,5],"tags":[],"class_list":["post-36","post","type-post","status-publish","format-standard","hentry","category-bash","category-python","category-reflective","category-webdev"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Reflecting on programming in 2023 - David Tang<\/title>\n<meta name=\"description\" content=\"Closing up the year with a reflection on what I have learnt with programming. My journey, lessons and takeaways in a quick bite format.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Reflecting on programming in 2023 - David Tang\" \/>\n<meta property=\"og:description\" content=\"Closing up the year with a reflection on what I have learnt with programming. My journey, lessons and takeaways in a quick bite format.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/\" \/>\n<meta property=\"og:site_name\" content=\"David Tang\" \/>\n<meta property=\"article:published_time\" content=\"2023-12-08T11:50:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-08T15:11:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ploomber.io\/images\/logo.png\" \/>\n<meta name=\"author\" content=\"David Tang\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"David Tang\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/\"},\"author\":{\"name\":\"David Tang\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/#\\\/schema\\\/person\\\/4038984c0f9c2f614e4fee857fd83037\"},\"headline\":\"Reflecting on programming in 2023\",\"datePublished\":\"2023-12-08T11:50:08+00:00\",\"dateModified\":\"2023-12-08T15:11:22+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/\"},\"wordCount\":783,\"image\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ploomber.io\\\/images\\\/logo.png\",\"articleSection\":[\"Bash\",\"Python\",\"Reflective\",\"Webdev\"],\"inLanguage\":\"en-GB\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/\",\"url\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/\",\"name\":\"Reflecting on programming in 2023 - David Tang\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ploomber.io\\\/images\\\/logo.png\",\"datePublished\":\"2023-12-08T11:50:08+00:00\",\"dateModified\":\"2023-12-08T15:11:22+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/#\\\/schema\\\/person\\\/4038984c0f9c2f614e4fee857fd83037\"},\"description\":\"Closing up the year with a reflection on what I have learnt with programming. My journey, lessons and takeaways in a quick bite format.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#primaryimage\",\"url\":\"https:\\\/\\\/ploomber.io\\\/images\\\/logo.png\",\"contentUrl\":\"https:\\\/\\\/ploomber.io\\\/images\\\/logo.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/2023\\\/12\\\/08\\\/reflect-programming-year\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Reflecting on programming in 2023\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/#website\",\"url\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/\",\"name\":\"David Tang\",\"description\":\"Python, R and programming in health research\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/#\\\/schema\\\/person\\\/4038984c0f9c2f614e4fee857fd83037\",\"name\":\"David Tang\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4fd7fb7d4052fc0d552fc28fe15aa04a16b2cb35b032cf621583104185b16328?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4fd7fb7d4052fc0d552fc28fe15aa04a16b2cb35b032cf621583104185b16328?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4fd7fb7d4052fc0d552fc28fe15aa04a16b2cb35b032cf621583104185b16328?s=96&d=mm&r=g\",\"caption\":\"David Tang\"},\"url\":\"https:\\\/\\\/blogs.imperial.ac.uk\\\/david-tang\\\/author\\\/dt20\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Reflecting on programming in 2023 - David Tang","description":"Closing up the year with a reflection on what I have learnt with programming. My journey, lessons and takeaways in a quick bite format.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/","og_locale":"en_GB","og_type":"article","og_title":"Reflecting on programming in 2023 - David Tang","og_description":"Closing up the year with a reflection on what I have learnt with programming. My journey, lessons and takeaways in a quick bite format.","og_url":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/","og_site_name":"David Tang","article_published_time":"2023-12-08T11:50:08+00:00","article_modified_time":"2023-12-08T15:11:22+00:00","og_image":[{"url":"https:\/\/ploomber.io\/images\/logo.png","type":"","width":"","height":""}],"author":"David Tang","twitter_card":"summary_large_image","twitter_misc":{"Written by":"David Tang","Estimated reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#article","isPartOf":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/"},"author":{"name":"David Tang","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/#\/schema\/person\/4038984c0f9c2f614e4fee857fd83037"},"headline":"Reflecting on programming in 2023","datePublished":"2023-12-08T11:50:08+00:00","dateModified":"2023-12-08T15:11:22+00:00","mainEntityOfPage":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/"},"wordCount":783,"image":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#primaryimage"},"thumbnailUrl":"https:\/\/ploomber.io\/images\/logo.png","articleSection":["Bash","Python","Reflective","Webdev"],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/","url":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/","name":"Reflecting on programming in 2023 - David Tang","isPartOf":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#primaryimage"},"image":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#primaryimage"},"thumbnailUrl":"https:\/\/ploomber.io\/images\/logo.png","datePublished":"2023-12-08T11:50:08+00:00","dateModified":"2023-12-08T15:11:22+00:00","author":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/#\/schema\/person\/4038984c0f9c2f614e4fee857fd83037"},"description":"Closing up the year with a reflection on what I have learnt with programming. My journey, lessons and takeaways in a quick bite format.","breadcrumb":{"@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#primaryimage","url":"https:\/\/ploomber.io\/images\/logo.png","contentUrl":"https:\/\/ploomber.io\/images\/logo.png"},{"@type":"BreadcrumbList","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/2023\/12\/08\/reflect-programming-year\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blogs.imperial.ac.uk\/david-tang\/"},{"@type":"ListItem","position":2,"name":"Reflecting on programming in 2023"}]},{"@type":"WebSite","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/#website","url":"https:\/\/blogs.imperial.ac.uk\/david-tang\/","name":"David Tang","description":"Python, R and programming in health research","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blogs.imperial.ac.uk\/david-tang\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/blogs.imperial.ac.uk\/david-tang\/#\/schema\/person\/4038984c0f9c2f614e4fee857fd83037","name":"David Tang","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/4fd7fb7d4052fc0d552fc28fe15aa04a16b2cb35b032cf621583104185b16328?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/4fd7fb7d4052fc0d552fc28fe15aa04a16b2cb35b032cf621583104185b16328?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4fd7fb7d4052fc0d552fc28fe15aa04a16b2cb35b032cf621583104185b16328?s=96&d=mm&r=g","caption":"David Tang"},"url":"https:\/\/blogs.imperial.ac.uk\/david-tang\/author\/dt20\/"}]}},"_links":{"self":[{"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/posts\/36","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/users\/1774"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/comments?post=36"}],"version-history":[{"count":5,"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/posts\/36\/revisions"}],"predecessor-version":[{"id":41,"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/posts\/36\/revisions\/41"}],"wp:attachment":[{"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/media?parent=36"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/categories?post=36"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.imperial.ac.uk\/david-tang\/wp-json\/wp\/v2\/tags?post=36"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}