So far, we've talked exclusively about things that happen in the user's browser. This is all fine and dandy, but suddenly you've realized that your fans (and a literal army of Russian bots) will not be denied their ability to leave you a message on the guestbook page of your website. Thanks to the direct brain infusion of knowledge you've gotten thus far, you feel pretty good about making a page that has a form on it. You can even check to see if it's blank and that it meets your rigorous standards for communication. But then you hit a wall...
Where the heck do I store the messages people send me on my website? Not just "how do I see the guestbook posts" but
"how do I AND everyone else see the same posts?" The answer is that somewhere, you need a persistence layer
, such as a
database, to store the data in a central place to be retrieved and displayed when people visit your site. The typical
way this Do-si-do happens is with some sort of back-end
or server-side
language that are exposed via APIs
that
query the database for you and return the appropriate data.
The terms back-end
and server-side
are pretty interchangeable, and they basically mean that there is code that is
being run on a server that you or your company own or rent. APIs
are the way that applications that commonly utilize
to interact with that server.
This level is going to cover a lot of ground, so maybe I should write some of the key concepts as Tweetable clickbait headlines...let's try:
- YOU WON'T BELIEVE WHAT THIS DEVELOPER DID WITH A FEW APIs, NODEJS, AND A DATABASE. THE ANSWER WILL SHOCK YOU!
- SCIENCE HAS DETERMINED THE WORLD'S BEST PROGRAMMING LANGUAGE, AND YOU'LL NEVER GUESS WHAT IT IS!
- THE WORLD'S MOST SUCCESSFUL COMPANIES SWEAR BY THIS TECHNOLOGY EVERY DAY.
More exciting? Maybe. Helpful? No. Let's just come to terms with the fact that we will have to get a little deeper, cover some concepts that may sound scary at first, and ultimately resolve ourselves to slowly peeling back the layers of mystery that exist around most backend technologies.
You've read this far. You can do it.
⚡️ APIs: Electric, Water, and Gas.
API
stands for Application Programming Interface
— but that doesn't matter unless you're on a first date and
really need a way to show the other person that you know how to party. APIs
are kind of like the utilities that flow
in and out of your house daily. In many cases, as you were building your house, you didn't set up your own hydroelectric
dam for electricity, you didn't drill into the earth for a natural gas supply, and the water flows magically
from...somewhere. You essentially plugged into the API
of whatever municipality you reside in to get you the things
you need to construct your house and make it functional.
An API's
basic functions are that an application somewhere makes a request
for a resource, and the server returns a
response
with the requested data upon success; or a descriptive error on failure. Many APIs
are stateless, meaning
that after the API
serves the data back, it's done —
basically like a Meeseeks from Rick and Morty.
This request/response
cycle is a fundamental element of most web applications. There are also other mechanisms, like
Socket
connections, that enable a connection to remain open for longer and have data pushed back to the client in
realtime, like in a chat application.
APIs
return data in a standard format so that your application knows how to parse the result. There are a few common
formats for an API
to return: XML
and JSON
.
XML
The XML
data format goes all the way back to 1998 and is meant to be readable by humans and computers. Here's an
example of what an XML
response for a house search might return...
<houses>
<house>
<address>123 Main Street</address>
<bedrooms>4</address>
<price>500999</price>
<hashottub>true</hashottub>
</house>
<house>
<address>221 Electric Avenue</address>
<bedrooms>5</address>
<price>620000</price>
<hashottub>false</hashottub>
</house>
<house>
<address>322 Elm Street</address>
<bedrooms>4</address>
<price>520000</price>
<hashottub>true</hashottub>
</house>
</houses>
You're going to notice that it resembles HTML
, with a hierarchy of nested open and close tags, and you're right —
since around the year 2000, most HTML
is a valid XML
document!
However, the key difference is that with HTML
, there are a predefined set of tags, whereas XML
tags can be
anything...even "hasHotTub
". In HTML
, the tags describe your document. In XML
, the tags describe your data.
Neat, you kinda see what XML
is, but now you're wondering how you...umm...get XML
to your application. Look no
further...
An example flow for your application would look like this:
- A user enters some search criteria and requests a list of houses in their area, in their price range, and at least 4 bedrooms. Hot tubs are not a dealbreaker.
- The server returns a list of houses and basically hangs up the call. And just like that telephone psychic you saw on TV the other night, it'll always pick up when you call again.
- Your application parses the
response
and displays the results to the user.
Great! That's the basic ins-and-outs of an API
that returns XML
. This is great and all, but if your application is
written in, say Javascript, you're going to have to take the raw XML
and parse it into a format that you can read and
manipulate with Javascript. There are literally hundreds of libraries to
do this. What if you didn't need to do this? What if your server could just return a format that could be read natively
by most modern web applications? Well, consider your wishes answered, and welcome my special guest JSON
...
🧠 Learn more about XML:
> https://www.youtube.com/watch?v=KeLiQXqVgMI
JSON
JSON
(pronounced JAY-SAWN) is a data format based on the Javascript
language's object
data type. It's really
light-weight and basically just consists of arrays (lists of stuff) and key-value pairs
, which are just a label for
the data, and then the data itself. Let's look at the same data from the XML
example in the JSON
format:
{
"houses": [
{
"address": "123 Main Street",
"bedrooms": 4,
"price": 500999,
"hashottub": true
},
{
"address": "221 Electric Avenue",
"bedrooms": 5,
"price": 620000,
"hashottub": false
},
{
"address": "322 Elm Street",
"bedrooms": 4,
"price": 520000,
"hashottub": true
}
]
}
Definitely still readable, right? Definitely a lot less repetition than XML
, with its opening and closing tags
repeated throughout. But, what's with all the weird curly braces and square brackets? Remember, this is
JSON: Javascript Object Notation
, and this means that this is a valid Javascript
object.
In Javascript
, an object is defined with { }
and objects can have keys
inside them to describe data. The example
object above has one key
named "houses." Just like with XML
, we can name these keys
to be anything we desire to
describe our data.
Keys
can contain a few different types of data:
- Booleans:
true
orfalse
- Strings: Any literal text in quotes – ****"roger", "1234 test street", "Des Moines", etc.
- Numbers:
1
,2
,3.1
,400000000000.24
, etc. - Arrays: Denoted with square brackets
[ ]
, arrays are a list of...things. Numbers. Booleans, Strings, Objects, even other Arrays! - Objects: A
key
inJavascript
can contain an even deeper object. In the example above, we have "address" as a string, but what if we did it as an object instead? We could!
{
"houses": [
{
"bedrooms": 5,
"price": 620000,
"hashottub": false,
"address": {
"houseNumber": 221,
"streetName": "Electric Avenue",
"city": "Boulder",
"state": "CO",
"zip"
}
},
//...etc
]
}
JSON
is pretty darn common as a data format in most web applications because it is:
- Light - less time spent transferring data
- Super readable - I find the extra tags in XML to be infuriatingly verbose
- Easily read - anything running
Javascript
can rip through bajillions of lines of aJSON
object without breaking a sweat.
Well, that's a nickel-tour of JSON
vs. XML
, which are without question the two dominant formats you'll see in any
modern web applications. Make sure you take time to understand how these look and how to describe data with them.
We know what an API
is, and we know a few of the formats that data can come back to us in...but did you know there are
even standards for how to request the data? Of course, there are...
🧠 Learn More about JSON:
> https://beginnersbook.com/2015/04/json-tutorial/
🧩 REST, GraphQL, WTF?
🍔 REST: GET me that cheeseburger, please.
REST
is one of those acronyms that feels like the US
government came up with, finding random words to match the zinger of a title they needed for some new legislation. That
important non sequitur out of the way, let's learn why REST
has been such a staple of the internet for creating,
reading, updating and deleting (aka CRUD
) data in applications.
Every time your browser visits a webpage on the internet, it makes a flurry of requests to various web servers to bring you the dopamine-inducing magic that is something like Twitter. You can see this if you open up the developer tools in your browser and check out the "network" tab.
See how the page slowly comes to life piece-by-piece? Those are the various components
that make up Twitter's user
interface, each fetching the data they need to present themselves. I put an arrow in the method
column to call out
something important, the keywords GET
or POST
next to each request. These are what are known as HTTP verbs
, and
they're at the core of REST
.
A few of the most commonly used verbs are as follows:
- GET - used to retrieve data. "Get me Tweets."
- HEAD - basically just asks for a status. "Are Tweets available?"
- POST - sends data to the server. "I'm submitting my first Tweet!"
- PUT / PATCH - overwrites/patches something that exists. "I need to edit that Tweet. That's not how you spell covfefe!"
- DELETE - deletes something on the server. "I'm going to take this Tweet down so the internet never ever sees it again."
Because you are into creating things that have totally never been done before, you have decided to build a Twitter
client to pull tweets into your website. Naturally, the first question is, "where do I begin?" A great place to begin
your journey is what is generally known as API Docs
— literally a map to all of the pieces of data that an API
makes
available. Any website or service with an API
available will generally have a "developer" section with tons of
documentation around how to get started with the API
.
Here's some of Twitter's API Docs
— check
it out. Do you notice all the verbs?
https://developer.twitter.com/en/docs/api-reference-index#twitter-api-v2
As you can see, there are a ton of different endpoints
that serve and accept data. Clicking on any of these will take
you to a more
descriptive page showing
you what data it accepts, a sample request, and then a sample response. Here's a sample response for getting a single
Tweet:
{
"data": {
"author_id": "2244994945",
"created_at": "2018-11-26T16:37:10.000Z",
"text": "Just getting started with Twitter APIs? Find out what you need in order to build an app. Watch this video! https://t.co/Hg8nkfoizN",
"id": "1067094924124872705"
},
"includes": {
"users": [
{
"verified": true,
"username": "TwitterDev",
"id": "2244994945",
"name": "Twitter Dev"
}
]
}
}
As you can see, you get a number of fields back in the response that tells you all you need to know about a single
Tweet. Well, what if you're really concerned about all that extra crap coming back, and you wanna just get back the
content of the Tweet? You can pass are a few optional parameters to the API
to slim it down, but ultimately it's up
to an API
developer somewhere to implement those toggles. This is one of the limitations of REST
- in general, you
can't customize the payload that comes back or even combine APIs
into one single response...and that's where something
awesome called GraphQL
comes in...
🧠 Learn more about REST:
> https://www.coursera.org/projects/restful-api-http-javascript
🍔 GraphQL: GET me that cheeseburger, please hold the pickles and lettuce.
GraphQL
is an open-source language for APIs
used internally in Facebook and then released to the public in 2015.
Without getting into too much detail, GraphQL
allows you to do some really cool things:
- Combine multiple
API
requests into a single shot. This would be like getting a user and their top friends in one shot instead of a waterfall of multiple calls that would traditionally happen. The fewer the requests, the faster the application. - Only return the data you need. Many
REST APIs
return a lot of extra data that you may not need. When you're dealing with users on mobile or limited bandwidth, every byte counts. - Have one singular
API
for your front-end app. WithGraphQL
, you can stitch together a myriad of various data sources into a singularAPI
for your application. If something changes in the way Twitter serves up theirAPI
, you can simply change your data source inGraphQL
to account for this, and your front-end code can pretty much stay the same.
Here's what a sample GraphQL
request for us to get just the Tweet content itself.
# POST REQUEST TO GRAPHQL SERVER
query getTweet {
{
data {
text
}
}
}
We would then get this back:
{
"data": {
"text": "Just getting started with Twitter APIs? Find out what you need in order to build an app. Watch this video! https://t.co/Hg8nkfoizN"
}
}
See that magic? You ask it for what you want, and it returns only what you need. As an avid collector of things from
across the internet, the one thing that has really stuck out with me about how to think about GraphQL
is an image of a
hamburger. I hope it provides the same synaptic connection for you...
🧠 Learn more about GraphQL
> https://graphql.org/learn/queries/
> https://www.toptal.com/api-development/graphql-vs-rest-tutorial
More in this series
Go get a coffee, then come right back, ok? I'll wait.
- Level 0: The Intro
- Level 1: HTML + CSS + Javascript - the 3 Amigos of the Web
- Level 2: Frontend Frameworks - Building blocks for your wildest dreams
- Level 3: 👈 YOU ARE HERE
- Level 4: Server-Side Languages - With great power comes great responsibility
- Level 5: Server-Side Frameworks - Trust me, you don't wanna write this crap from scratch.
- Level 6: Databases - Your Excel sheet in the sky.
- Level 7: How to internet - Domains & DNS, Hosting, The Cloud™, & Serverless
- Level 8: Authentication and Authorization - House keys vs. giving your neighbors a temporary combination to your garage.
- Level 9: Content Management Systems - A fill-in-the-blank adventure!
- Level 10: Working With Engineers: Happiness, Agile, and Git
- Level 11: Working With Engineers: Time, Team Chemistry and Van Halen
- Level 12: Working With Engineers Tracer Bullets, Team Sizes, and Making Space.