Web Technologies

Puppeteer.JS – Using Headless Chrome for Site Crawling

PuppeteerJS essentially allows you to automate Chrome.
Headless Chrome allows you to run Chrome without actually rendering the webpage. Sounds silly, but has a lot of useful applications, you could for example simply write a test script that ensures that your website is still working correctly.

Installation

npm i puppeteer
# or
yarn add puppeteer

Usage

We are going to look at a quick example of how to log In to a site and then do some operation.

Inititalize Puppeteer

You need to run it in an async function, simply because you do not know how long it will take until chrome has started.
so with

We start our browser. The flag headless is set to ‘true’ as default, however for debugging purposes, you should set it to ‘false’;

Login

To Login into the site we need three things:
* The URL for the Login Page
* CSS Selector for the Username Field
* CSS Selector for the Password Field

To obtain the the Selectors you can use the Chrome DevTools (F12). Simply select the HTML Field and with Rightclick select Copy Selector.

Fetch all Links

Now since you are logged in to the site, you can navigate to any site and fetch all the links.

Final Code

Posted by happyneal in Blog, Web Technologies, 0 comments

WebDev – Articles – 02

Articles

The engineer’s guide to not making your app look awful

4 Surprising Reasons to invest in UX

ReactJS

The licencing of React will change with Version 16 it will be a MIT licence.
Release of ReactJS 16

Java

IBM J9 joins Eclipse Foundation as OpenJ9
In essence OpenJ9 is an Alternative to the OpenJDK you can use for enterprise Applications.

Oracle Java EE joins Eclipse Foundation
While the exact date when Java EE will join the Eclipse foundation is not known. The move will happen some time after the completion of Java EE 8.

Tools

Material Palette

A website to test out various color shemas for your Material UI based Design
Similar to Adobe Kuler

Atom IDE

To play around with this new Package you need Version 1.21 (Currently Beta): https://atom.io/beta

Announcement

Posted by happyneal in Web Technologies, 0 comments
VSCode: Launch create-react-app and Chrome with launch.json

VSCode: Launch create-react-app and Chrome with launch.json

Developing React (with create-react-app) and Visual Studio Code you usually press ESC-` and then npm start.
The script from create-react-app then automatically starts a Browser. That you then close.
hen reopen by pressing F5 to start Chrome in debug mode.

Let’s make these steps a little quicker.

Create a Task for npm start

Press Ctrl-Shift- and Select “Tasks: Configure Default Test Task”
This will create a tasks.json file.

In the tasks.json file you need to add a couple of values:
* isBackground: true – launch.json will not wait untill the task completes
* problemMatcher Needs to be defined to figure out when the task has completet its initialisation phase and it is safe to continue with the next task

Configure create-react-app

To prevent launching the browser you need to add in your .env-file following line:

BROWSER=none

More Info:
* .env-file

Configure the Launch.json file

Press F5 and select Chrome and a launch.json file will be created.
* Change the port to 3000 (create-react-app default)
* Add a preLaunchTask to start the task we defined earlier

Start Working

Tadaa, now you press F5 and can start debugging directly in vscode. The background task will continue running even when you stop debugging.

Stop Working

You need to terminate the task via ctrl-shift-p > terminate Task. (Or you just close vscode)

Posted by happyneal in Blog, Web Technologies, 0 comments

DEV281x: Introduction to ReactJS

I took a quick look at Microsoft Introduction to ReactJS, not to actually learn React, but to evaluate how good the materials are for a beginner to learn React.

tl;dr
Avoid the assessment. Maybe use create-react-app instead of codepen.
Covers the basics quite well.

Lesson 01 | JSX and React Components

Instead of using “create-react-app” as a starting point, the course tells you to go to codepen to check out react.

I guess that makes it a little more focused on the React concepts and less focused on all the surrounding technologies that are needed in an production environment.

After that it covers the basics of ReactJS, and JSX.

If you are a beginner you probably want to check out create-react-app.

Lesson 02 | State, Life Cycle and Event Handlers

I think the chapter covers the basics quite well. Nothing really to add here.

Lesson 03 | Lists and Forms

Here i think it is very nice that they cover the basics of handling multiple Form Elements with a generic handler.

Assessment

The Assessment is quite theoretical and asks rather complicated questions. These questions are especially hard if you have already worked with React as you would never even try to build something in that way.

As a beginner I would completely skip the Assesment. It does not help you to further your understanding of React. It actually also confronts you with bad practices, while inferring that this would be the way how you would actually do things

Conclusion

If you use create-react-app with vscode I think you would have a much better starting experience.
The course is great to get started with ReactJS, if you avoid the assesments and rather just experiment with your own code base.

Posted by happyneal in Programming, Web Technologies, 0 comments

package.json: Updating Fixed Versions with npm-check

One of the common problems when running a larger project is that you need to use fixed versions in your package.json file. But at the same time you need to regularly update your packages.
The most elegant way is using npm-check. The small tool allows you to select which packages should get an update and update accordingly.

Installation

npm i -g npm-check

Usage

To update the packages in your project you now simply run npm-check -u. If you want to ensure that you are installing the exact package run it with the additional optional flag -E to ensure exact-versions.

npm-check -u -E

With Space you select the packages and with Enter you install the package.

Posted by happyneal in Blog, Web Technologies, 0 comments

Disable the package-lock.json file

With NPM 5, npm has started to create a “package-lock.json” file.
It ensures that some dependency tree is identical on every developers environment. Official Documentation.

Now, that may be important on some projects. In my personal projects I do not really care. And in the professional projects I did not have any issues how things were done before.

Actually with the new way of doing things, you must first remove the package-lock.json file and then run “npm update” to install newer versions of stuff. Then again, this can cause problems as when some other person also creates the package-lock file, or a merge conflict occurs etc. In the end totally defeating the purpose of the file.

Disabling the Package Lock File for a Project

  1. Create a file called .npmrc
  2. Open the file and add the line package-lock=false

Global Disable

If you currently want to disable the behavior on all projects run

npm config set package-lock false

In the future the feature may be more easy to use. For now I at least will stick to the old way of how to do things.

Posted by happyneal in Web Technologies, 0 comments