Loading websites into Brunch

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email

Building Brunch was not easy but it was fun and challenging. We had to be creative and find solutions to some pretty difficult problems. I’m gonna talk here about one of the bigger ones: how do we load the websites into the app and interact with them?

When you use Brunch, you’re interacting with the live website, loaded inside an iframe. Iframe?! Yes, you heard that right 😅. If you’re a web developer, you know that iframes are tricky to work with. A lot of websites can’t be loaded inside an iframe and they are pretty limited in terms of accessing the iframe’s content.

In this article, I’m gonna briefly explain how we managed to solve this and how many times I’ve coded this service from scratch (yes, a lot of times you don’t get it right from the first shot).

When we started Brunch, I had two ideas:

  • proxy the website and all it’s resources
  • clone the website

Why clone/proxy the website, you may wonder. We needed to do this to make sure the web page and all it’s resources are loaded correctly and also to modify the HTML and inject some Javascript code so we can communicate with the page from outside the iframe.

Iteration 1 – Proxy the website

When we started, we decided to go with option 1 – proxy the website. This way we don’t need to host a lot of websites and also we could display the live website (in case some changes were made after the website was added to the app).

We coded this and it was working ok-ish. I mean average loading speed and average “websites loading correctly”.

We tried to refactor and improve this but we ended up at a point where we fixed something and broke something else. At this point, we decided to try the second approach.

Iteration 2 – Clone the website

We implemented a cloning service that was saving and serving the websites from a CDN. It looked promising but it had one major problem: the first loading time for a website was way too big and users complained about it.

The advantage of this method was that websites were loading correctly most of the time. So, we decided to combine those two methods into one.

Iteration 3 – Proxy and cache

The method that we are using now is when you try to open a website into Brunch, it will try and serve that from the cache. If we have the website cached, will serve it from CDN. If not, we’ll proxy the live website and cache the website in the background. This way you’ll get decent loading speed even when the website is not cached, and great speed when the website is cached.

It may sound that we implemented these 3 methods in 3 days but it actually was over one year ⏰.

Now, that I’ve explained to you how are we loading the websites into Brunch, let’s get back a little to why we do all this. We wanted to inject some Javascript into the loaded website so we can establish a two-way communication channel between the website and Brunch. Luckily, there’s a web API for this and we can take advantage of window.postMessage().

The window.postMessage() method safely enables cross-origin communication between objects; e.g., between a page and a pop-up that it spawned, or between a page and an iframe embedded within it.

Example

The example was taken from developer.mozilla.org.

postMessage example
postMessage example

This way, we could easily send an event to Brunch when a user clicks an element from the iframe so we could initiate the add feedback process. Or, we could highlight the current element using the hover event.

If you have any questions or you want to reach me, check out my LinkedIn.

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email
Interested in investing in Brunch?
This is default text for notification bar