Puppeteer.JS – Using Headless Chrome for Site Crawling

PuppeteerJS essentially allows you to automate Chrome.
Headless Chrome allows you to run Chrome without actually rendering the webpage. Sounds silly, but has a lot of useful applications, you could for example simply write a test script that ensures that your website is still working correctly.


npm i puppeteer
# or
yarn add puppeteer


We are going to look at a quick example of how to log In to a site and then do some operation.

Inititalize Puppeteer

You need to run it in an async function, simply because you do not know how long it will take until chrome has started.
so with

We start our browser. The flag headless is set to ‘true’ as default, however for debugging purposes, you should set it to ‘false’;


To Login into the site we need three things:
* The URL for the Login Page
* CSS Selector for the Username Field
* CSS Selector for the Password Field

To obtain the the Selectors you can use the Chrome DevTools (F12). Simply select the HTML Field and with Rightclick select Copy Selector.

Fetch all Links

Now since you are logged in to the site, you can navigate to any site and fetch all the links.

Final Code

WebDev – Articles – 02


The engineer’s guide to not making your app look awful

4 Surprising Reasons to invest in UX


The licencing of React will change with Version 16 it will be a MIT licence.
Release of ReactJS 16


IBM J9 joins Eclipse Foundation as OpenJ9
In essence OpenJ9 is an Alternative to the OpenJDK you can use for enterprise Applications.

Oracle Java EE joins Eclipse Foundation
While the exact date when Java EE will join the Eclipse foundation is not known. The move will happen some time after the completion of Java EE 8.


Material Palette

A website to test out various color shemas for your Material UI based Design
Similar to Adobe Kuler

Atom IDE

To play around with this new Package you need Version 1.21 (Currently Beta): https://atom.io/beta


So good they can’t ignore you

Notes on the Book So good they can’t ignore you by Cal Newport


Rule 1

Don’t follow your passion

Rule 2

Be so good they can’t ignore you (Importance of Skill)

Rule 3

Turn down the Promotion (Importance of Control)

Rule 4

Think Small, Act Big (Importance of Mission)

Don’t follow your passion

Steve Jobs story did not start with computers, however over time he became passionate about it

  • Passion is Rare
  • Passion Takes Time
  • Passion is a Side Effect of Mastery

Be so good they can’t ignore you

(Importance of Skill)

Adopting the Craftsman Mindset

Tice is willing to grind out long hours with little recognition, but

That’s because it’s in service to

Something he’s obviously passionate about and

Has been for a long time. He’s found that one job

That’s right for him

The Power of Career Capital

If you want a great job, you have to build up rare and valuable skills – career capital

Passion mindset – what can the world offer me

Craftsman mindset – what can I offer the world

Traits that define good Work

  • Creativity
  • Impact
  • Control

Three Disqualifiers for applying the craftsman mindset

  • The job presents few opportunities to distinguish yourself by developing relevant skills
  • The job focuses on something you think is useless or actively bad for the world
  • The job forces you to work with people you really dislike

Becoming a Craftsman

Deliberate Practice

If you just show up and work hard, you’ll soon hit a performance plateau beyond which you fail to get any better.

To successful adopt the craftsman mindset, we have to approach our jobs in the same way as Garry Kasparov his chess training – with a dedication to deliberate practice.

Five Habits of the Craftsman

  1. Decide in What kind of Capital Market you are in
  2. Identify your Capital Type
  3. Define “Good”
  4. Stretch and Destroy
  5. Be Patient

Turn down the Promotion

(Importance of Control)

Control without career capital is not sustainable.

The point at which you have acquired enough career capital to get meaningful control over your working life is exactly the point you‘ve become valuable enough to your current employer will prevent you from making the change.

The Law of Financial Viability

When deciding whether to follow an appealing persuit that will introduce more control to your worklife, seek evidence of whether people are willing to pay for it. If you find this evidence continue. If not, move on.

Think Small, Act Big

(Importance of Mission)

The Law of Remarkability

For a mission-driven project to succeed, it should be remarkable in two different ways. First, it must compel people who encounter it to remark about it to others. Second, it must be launched in a venue that supports such remarking.

WebDev – Articles – 01

A short summary of some of the interesting articles I came across in the last couple of weeks.


Most Important Color in UI Design
A very interesting article on the Color Blue and why it is used so much.


Canary 62 Dev Tools

One of the highlights of the new Dev tools is that you can capture screenshots of a specific node.

  1. Select the Node
  2. Open the Command Menu (CTRL-SHIFT-P)
  3. Type ‘capture node’


VSCode 1.16

  • HTML now finally gets autoclose tag. However it is not enabled for JSX, so you still need to install the Auto-Close-Plugin
  • Support of Typescript 2.5
  • Faster Refactorings for Typescript, by selecting a Code Segment you can rightclick and say “Extract Function”



A useful tool to detect which technologies are used by a specific website.


Edge Insecure by Design: Bypass Content Security Policy (CSP)
A reported Security Issue that effects Safari, Chrome and Edge, got fixed in Safari and Chrome. Microsoft responds: Works as Designed.

Random Articles

Trending Developer Skills based on Job Descriptions
An interesting way to take a look at the vacant Job Positions and concluding which technologies are currently being used for the Production Stack.

(tl;dr: Rapid Rise of React, NodeJS and Postgres)

Why Coding Style Matters
A short article about the benefits of a clear and consistent coding style.ama
“Programs are meant to be read by humans and only incidentally for computers to execute.”
— H. Abelson and G. Sussman (in “Structure and Interpretation of Computer Programs”)

VSCode: Launch create-react-app and Chrome with launch.json

Developing React (with create-react-app) and Visual Studio Code you usually press ESC-` and then npm start.
The script from create-react-app then automatically starts a Browser. That you then close.
hen reopen by pressing F5 to start Chrome in debug mode.

Let’s make these steps a little quicker.

Create a Task for npm start

Press Ctrl-Shift- and Select “Tasks: Configure Default Test Task”
This will create a tasks.json file.

In the tasks.json file you need to add a couple of values:
* isBackground: true – launch.json will not wait untill the task completes
* problemMatcher Needs to be defined to figure out when the task has completet its initialisation phase and it is safe to continue with the next task

Configure create-react-app

To prevent launching the browser you need to add in your .env-file following line:


More Info:
* .env-file

Configure the Launch.json file

Press F5 and select Chrome and a launch.json file will be created.
* Change the port to 3000 (create-react-app default)
* Add a preLaunchTask to start the task we defined earlier

Start Working

Tadaa, now you press F5 and can start debugging directly in vscode. The background task will continue running even when you stop debugging.

Stop Working

You need to terminate the task via ctrl-shift-p > terminate Task. (Or you just close vscode)

DEV281x: Introduction to ReactJS

I took a quick look at Microsoft Introduction to ReactJS, not to actually learn React, but to evaluate how good the materials are for a beginner to learn React.

Avoid the assessment. Maybe use create-react-app instead of codepen.
Covers the basics quite well.

Lesson 01 | JSX and React Components

Instead of using “create-react-app” as a starting point, the course tells you to go to codepen to check out react.

I guess that makes it a little more focused on the React concepts and less focused on all the surrounding technologies that are needed in an production environment.

After that it covers the basics of ReactJS, and JSX.

If you are a beginner you probably want to check out create-react-app.

Lesson 02 | State, Life Cycle and Event Handlers

I think the chapter covers the basics quite well. Nothing really to add here.

Lesson 03 | Lists and Forms

Here i think it is very nice that they cover the basics of handling multiple Form Elements with a generic handler.


The Assessment is quite theoretical and asks rather complicated questions. These questions are especially hard if you have already worked with React as you would never even try to build something in that way.

As a beginner I would completely skip the Assesment. It does not help you to further your understanding of React. It actually also confronts you with bad practices, while inferring that this would be the way how you would actually do things


If you use create-react-app with vscode I think you would have a much better starting experience.
The course is great to get started with ReactJS, if you avoid the assesments and rather just experiment with your own code base.

VSCode Extensions (August 2017)

Here are a couple of VSCode Extensions I currently am using.


Script to install all suggested Plugins

Installing Extensions

There are two ways of installing extensions. Either in the editor or via command line.


  1. Press Ctrl-P
  2. Paste Code like ext install eamodio.gitlens
  3. Click on install

Command Line

  1. Open command line
  2. Enter command code --install-extensions eamodio.gitlens

Markdown All in One

A litte tool that makes it a little easier to format Markdown documents
Adds useful shortcuts like ctrl-b to make something bold.

code --install-extension yzhang.markdown-all-in-one

Auto Close Tag

If you do not use Emmet shorthand, it is very useful of closing your html tags.

Sadly the plugin cannot figure out if you are defining a type in Typescript and also attempts to close the casting tags.

code --install-extension formulahendry.auto-close-tag

Auto Rename Tag

This plugin takes care of the closing tag of your html-tag. It also works for JSX&TSX files.

code --install-extension formulahendry.auto-rename-tag


A great extension to vastly improve your git experience in vscode

code --install-extension eamodio.gitlens

TODO Highlight

Well it Highlights Todos… so well thats good.
You can extend it to display any type of label in your comments.

Plus it can generate a handy list of all todos in the project.
(Sadly jumping to the TODO in the List does not work in Ubuntu)

code --install-extension wayou.vscode-todo-highlight

Path Intellisense

Well it detects when you are trying to define a path to a file and autocompletes it for you.

In typescript it also correctly removes the file extension for imports.

code --install-extension christian-kohler.path-intellisense