Posted in 2017, conference, JavaScript, webrtc, websc

WebRTC – architecture & protocols

In this blog post I shall discuss the architecture & protocols powering WebRTC. This blog series is for my upcoming WebRTC workshop at the Web Summer Camp, Croatia 2017.

While WebRTC has greatly simplified real time communication on the web through the browser, it’s background comprises of a collection of standards, protocols, and JavaScript APIs! The power of WebRTC is such that only a dozen lines of JavaScript code and any web application can enable peer-to-peer audio, video, and data sharing between browsers (peers).

The Architecture

Screen Shot 2017-08-19 at 13.24.23
WebRTC Architecture

WebRTC architecture consists of over a dozen different standards, covering both the application and browser APIs jointly operated by WEBRTC – W3C Working Group and RTCWEB – IETF Working Group. While its primary purpose is to enable real-time communication between browsers, it is also designed such that it can be integrated with existing communication systems: voice over IP (VOIP), various SIP clients, and even the public switched telephone network (PSTN), just to name a few.

WebRTC brings with it all the capabilities of the Web to the telecommunications world, a trillion dollar industry!

Voice and Video Engines

Enabling RTC requires that the browser be able to access the system hardware to capture both voice and video. Raw voice and video streams are not sufficient on their own. They have to be –

  1. Processed for noise reduction and echo cancellation
  2. Automatically encoded with one of the optimized narrowband or wideband audio codecs
  3. Used with a special error-concealment algorithm to hide the negative effects of network jitter and packet loss

Sender

  1. Process the raw stream to enhance quality
  2. Synchronize and adjust the stream
    • to match the continuously fluctuating bandwidth and latency between the clients

Receiver

  1. Decode the received stream in real-time
  2. Adjust the decoded stream to network jitter and latency delays
Screen Shot 2017-08-19 at 14.00.17
Voice and Video Engines

The fully featured audio and video engines of WebRTC take care of all the signal processing. While all of this processing is done directly by the browser, the web application receives the optimized media stream, which it can then forward to its peers using one of the JavaScript APIs!

VoiceEngine is a framework for the audio media chain, from sound card to the network.

VideoEngine is a framework for the video media chain, from camera to the network, and from network to the screen.

Audio Codecs

Screen Shot 2017-08-19 at 14.24.25
wideband audio
  1. iSAC: A wideband and super wideband audio codec for VoIP and streaming audio. iSAC uses 16 kHz or 32 kHz sampling frequency with an adaptive and variable bit rate of 12 to 52 kbps.
  2. iLBC: A narrowband speech codec for VoIP and streaming audio. iLBC uses 8 kHz sampling frequency with a bitrate of 15.2 kbps for 20ms frames and 13.33 kbps for 30ms frames.
  3. Opus: Supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s. Opus supports frame sizes from 2.5 ms to 60 ms, and various sampling rates from 8 kHz (with 4 kHz bandwidth) to 48 kHz (with 20 kHz bandwidth, where the entire hearing range of the human auditory system can be reproduced).

Video Codecs (VP8)

  • The VP8 codec used for video encoding requires 100–2,000+ Kbit/s of bandwidth, and the bitrate depends on the quality of the streams.
  • This is well suited for RTC as it is designed for low latency.
  • This is a video codec from the WebM project.

Real-Time Network Transports

Unlike all other browser communication which use Transmission Control Protocol (TCP), WebRTC transports its data over User Datagram Protocol (UDP).

The requirement for timeliness over reliability is the primary reason why the UDP protocol is a preferred transport for delivery of real-time data.

  • TCP delivers a reliable, ordered stream of data. If an intermediate packet is lost, then TCP buffers all the packets after it, waits for a retransmission, and then delivers the stream in order to the application. 
  • UDP offers no promises on reliability or order of the data, and delivers each packet to the application the moment it arrives. In effect, it is a thin wrapper around the best-effort delivery model offered by the IP layer of our network stacks.
Screen Shot 2017-08-19 at 15.09.13
WebRTC network protocol stack

UDP is the foundation for real-time communication in the browser. In order to meet all the requirements of WebRTC, the browser needs a large supporting cast of protocols and services above it to traverse the many layers of NATs and firewalls, negotiate the parameters for each stream, provide encryption of user data, implement congestion and flow control, and more!

The RTP Stack

  1. ICE: Interactive Connectivity Establishment
  2. STUN: Session Traversal Utilities for Network Address Translation (NAT)
  3. TURN: Traversal Using Relays around NAT
  4. SDP: Session Description Protocol
  5. DTLS: Datagram Transport Layer Security
  6. SCTP: Stream Control Transport Protocol
  7. SRTP: Secure Real-Time Transport Protocol
  • ICE, STUN, and TURN are necessary to establish and maintain a peer-to-peer connection over UDP.
  • DTLS is used to secure all data transfers between peers; encryption is a mandatory feature of WebRTC.
  • SCTP and SRTP are the application protocols used to multiplex the different streams, provide congestion and flow control, and provide partially reliable delivery and other additional services on top of UDP.
  • Session Description Protocol (SDP) is a data format used to negotiate the parameters of the peer-to-peer connection. However, the SDP “offer” and “answer” are communicated out of band, which is why SDP is missing from the protocol diagram.

 

 

Advertisements
Posted in 2017, conference, JavaScript, webrtc, websc

WebRTC – a detailed history

The Past

The need to connect virtually and have video conferences and communications on the web has been around for a while. In the past, Flash was one of the popular ways to achieve this. The alternate to this was plug-ins or an installable application on the PC. From a user’s perspective, all these methods required additional installations. From a developer’s perspective, they had to study complex stack and protocols.

The birth of WebRTC

WebRTC technology was first developed by Global IP Solutions (or GIPS), a company founded around 1999 in Sweden. In 2011 GIPS was acquired by Google and the W3C started to work on a standard for WebRTC. Since then Google and other major players in the web-browser market, such as Mozilla and Opera, have been showing great support for WebRTC.

Screen Shot 2017-08-06 at 12.12.30

The newly formed Chrome WebRTC team focused on open sourcing all the low level RTC components such as codecs and echo cancellation techniques. The team added an additional layer – a JavaScript API as an integration layer to web browsers. By combining these audio and video components with a JS interface, this spurred innovation in the RTC market.

A few lines of JS code and no licensing, integration of components or deep knowledge of RTC!

WebRTC – A Standard

WebRTC is a standard for real-time, plugin-free video, audio and data communication maintained by –

  • IETF – defines the formats and protocols used to communicate between browsers
  • W3C – defines the APIs that a Web application can use to control this communication

WebRTC is a standard that has different implementations

WebRTC is a standard that has different implementations, such as OpenWebRTC and webrtc.org. The initial version of the OpenWebRTC implementation was developed internally at Ericsson Research. The latter is maintained by the Google Chrome team.

Cover image for this post is from here.
Posted in Outreachy, Random

Biannual updates

I like to set my goals at the beginning of every year and reflect on them about what’s being achieved and what isn’t. Here is what I have written in the past:

For 2017, this is what I had set.

goals-1
2017 goals

Having achieved most of them already, I thought of a biannual update for this year. And today being 31.05.2017, it is a good time of the year to share these updates.

Although Outreachy commenced officially yesterday, me and Bianca have already started our Lightbeam project and we are two weeks through. It’s been fun two weeks getting to learn lot of things, interacting with mentors and working remotely with people from different time-zones.

My next blog post will be on the test environment setup for the Lightbeam web extension. I am super happy for being able to set this up from scratch and I can blabber about this on & on 😉

My personal website is a work in progress and I intend to launch it soon. It features the TIL series.

Ending this post with words of wisdom to stay focused to be able to achieve your goals:

  • PRIORITISE things
  • Have PATIENCE
  • CHALLENGE yourself

See you 🙂

Posted in Random

Mentoring @ Hackathon

Recently I had this opportunity to be a mentor at a hackathon. It was a Health & Fitness hackathon which took place on 14-16 October, 2017. The hackathon was organised by Geek Girls Carrots Berlin and my employer Zalando was one of the sponsors.

This slideshow requires JavaScript.

 

I was contacted by the employer branding team within Zalando to participate as a mentor and I had an amazing weekend. Although I wanted to participate as a participant at this hackathon, the idea of being a mentor was overwhelming and I jumped in to volunteer.

My colleague Andra accompanied me on Day 1, Day 2 and Iuliia accompanied me on Day 3.

Day 1: 14.10.2016

It was Friday evening but I was gladdened to watch out the participant turn-out. It was a decent number of audience. The evening was meant to meet & greet mentors and participants. After a formal welcome, mentors introduced themselves and participants were asked to brainstorm on their Health & Fitness related hacks for the weekend and build teams.

I met couple of women who were very enthusiastic about their participation. Me and Andra tried to give them insights about being able to use the Zalando Shop API for this hackathon.

The evening was accompanied with good, healthy food and we were hosted at the Native Instruments, Berlin office.

Day 2: 15.10.2016

It was a pleasant Saturday morning and the previous evening made sure I had my motivation level turned on to show up at the venue on time.

20161016_151737

I was excited to talk about the Zalando Shop API. Since it was a health & fitness hackathon, Zalando’s fashion reputation was less enticing amongst the participants. Our pitch was to lure the audience by asking them to come up with ideas which involved a little bit of shopping. After all, you need new clothes, accessories etc. for a healthy start at the gym 😉

We offered them 20% discount code for shopping at Zalando. Here is the voucher code for all the readers – ZRNN48454E. Happy shopping!

The API briefing session was followed by a VR workshop using the Unity Framework and the hands-on-hacking session followed thereafter.

One of the teams reached out to me and Andra asking for some help in getting started with React.

We were hosted at the Axel Springer, Berlin office. The food served was delicious. The rest of the day was intense hacking session and the participants were engrossed in their hacks. True to the hackathon’s theme, there was a Zumba workout session in the evening 🙂

picsart_10-16-06-15-27

Day 3: 16-10-2016

Mentor participation was required only during the second-half of this day for the jury discussions and voting. I decided to quickly run my weekend errands at home. Meanwhile I was helping one of the teams remotely with their React doubts.

Around 2:00 pm, it was jury time. I and Iuliia were quite excited about this. Being one of the jury member was one of my best experiences.

There were 7 teams all set to present their hacks. The source-code of all the teams can be found here.

I was amazed at the energy level of all the participants. There was no drop in their energy, inspite it being a Sunday. The presentations were excellent. I enjoyed listening to every detail. Me and Iuliia were silently making our notes and voting them based on code, idea and potential behind the hack.

After the presentations, all the jury assembled in one room. We were voting and it was interesting how the votes differed based on different evaluation criteria.

First criteria – potential and uniqueness of the hack.

Second criteria – presentations and team-spirit.

Third criteria – Implementation (working prototype)

It was surprising that looking into the source-code was the last criteria and the team ratings varied when this happened.

We had the winners listed at the end and this whole experience was elating.

picsart_10-16-06-27-24

Last but not the least, it was a weekend well spent 🙂 Mentors and participants had this as a take-away gift!

picsart_10-16-06-09-54

Here is the short-video featuring the hackathon

Posted in JavaScript

JSUnconf 23-24 April 2016, Hamburg

Disclaimer: Opinions expressed here are my own! Pictures are copied from twitter @jsunconf.

Now start reading…

It’s day #2 at JSUnconf, Hamburg and it’s snowing here! It’s weekend, yet nobody is tired. The enthusiasm within the audience/community is crazy. Everybody is here to share and learn something new.

jsunconf-audience

This was my first time at an Unconf! Unlike other conferences, the good thing about Unconfs is that the audience get to vote for the talk proposals on the fly. This is absolutely a conference for the people and by the people.

The two days started with a healthy breakfast and a quick welcoming of the audience. This was followed by potential speakers proposing their ideas. Voting was followed next and the schedule for the respective days was released.

As a speaker, I feel this is an interesting concept to validate your talk proposal. The joy of getting votes and getting a chance to speak is cosmic. My talk did make it to stage on day #1. My slides can be found here. The talk was on Natural User Interfaces using JavaScript. I had a chance to validate my idea before going live at JSConfBP, Budapest.

My session was at 14:30 on day #1. With 3 parallel sessions at a given time, I had a decent audience count listening to me. The demos which I had prepared, showed up very well and all the hard work paid off.

Being a foodie, it would be unfair from my part if I don’t post some FOOOOOOOOOD pictures 😉

JSUnconf highlights

  • It was a well-defined and nicely emphasised code of conduct.
  • This is what was there at both the rest-rooms. Promoting diversity at it’s best!jsunconf-all-gender
  • The coffee cups were re-usable.
  • The diversity scholarship of 300€ was awarded to 17 deserving candidates. I was one of them.jsunconf-diversity

From annoying pop-ups to JS on the back-end, the JavaScript community has seen JS in its varied forms. Conferences like these are a great place to network, learn and share knowledge.

My personal favorite talk at JSUnconf was about CSS refactoring by Christoph Reinartz, Trivago (@pistenprinz). Even though it was more about CSS, talks like these come out with a strong message about the need to use design patterns, code refactoring, terms much reserved to back-end and eyes drooling when the front-end community talks about.

It’s 14:15 (day #2) as I am ending this write-up. With 3 more hours to go here at the conference, it was a weekend well-spent. I have proposed a lightning talk too and is due in next 90 minutes. I made some new friends as well!

jsunconf-new-friends

Long live #javascript!