The proper design process in web development
Published on 2023-11-04. Modified on 2023-11-09.
Table of content
- Not everyone has fast Internet and fast computers
- Performance, security, convenience and principles in relation to software development
- Getting the balance right
- The decision making process
- Final comments
- Recommended reading
I know all about fast deployment and fast shipping and the constant fear of someone else eating your lunch, but there is also such a thing as something being too crappy to ship.
The monetized part of the software industry sucks today. Not only in web development, but in software development in general. All that these companies care about is how much money can be made as quickly as possible, even when that mean that they ship total crap. In e.g. game development it has become a standard that games are rushed and released with loads of errors and performance problems.
If we should transfer the current status of software and web development to the car manufacturing industry, then it would be comparable to releasing cars with engines that don't run correctly.
"What is that you say? The ignition timing is out of sync? The gearbox is not properly aligned with the engine causing the shaft from the flywheel to the gearbox to bind? The airbag inflator module deploys too slow? Oh, well, the deadline is tomorrow, ship it!"
While there is nothing wrong in trying to optimize the development process, the frantic obsession with fast delivery has become a really big problem. Too many corners are being cut.
It doesn't help either that all these companies who sleeplessly worry about fast delivery are being taken advantage of by a cultist crowd of people who religiously promote different software development methodologies, most of which are useless junk. A lot of conferences, a lot of books, and a lot of disciples. Hoards of money grabbing snake oil selling evangelists. They make endless promises of improved development and project managers eat all of it raw in their desperate attempt to fulfill the requirements of greedy business owners and investors.
We have all had the unpleasant experience of buying a product or a service that was bad, yet still pushed to marked despite flaws. So, if we have had that experience, should we not try to do better ourselves?
Not everyone has fast Internet and fast computers
I currently have a fibernet based Internet connection with good speed, but not long ago I only had access to the Internet via a DSL modem running on an old copper based phone line. It was not only slow, but the entire family had to share it, and the copper wires were so poorly insulated against electromagnetic noise that it randomly lost connection to the ISP. Every time it did that, it took about 2 minutes before it regained connection again.
Many people in Denmark today still only have access to the Internet via DSL modems. In some few parts of Denmark you can't even get that.
According to International Telecommunication Union (ITU), as of 2021, 4.9 billion people, actively use the internet. About 2.9 billion people are still offline, of which an estimated 96 per cent live in developing countries. And even among the 4.9 billion counted as active "Internet users", many hundreds of millions only get the chance to go online infrequently, via shared devices, or using connectivity speeds that markedly limit the usefulness of their connection.
I know two families who live in Pakistan. Both families are regularly without electricity for hours because the matter of balancing the country's supply against the demand for electricity is unresolved. When they do have electricity, they need to use it sparingly. This means that people gather around a simple old laptop hoping to brush up on news, study, or get a glimpse of something exciting on the Internet.
I also know of other people around the world, in the US, Belgium, Scotland, England, Australia, Germany, Poland, and many other places, that do not have access to fast Internet.
In the US people typically pay twice as much as Europe for high speed Internet, when they can even get it. Across the country, internet service providers are simulating that there is a major competition and are keeping high bandwidth Internet back. It's a nationwide problem, even in Silicon Valley where you'd think Internet might be great.
I pay $100 a month for Internet. My husband, a video producer, can't upload his videos to the company's servers and there's no point in the kids even trying to stream games to YouTube or Twitch.
― Internet user from the US
It's a humbling experience when you are used to fast Internet to browse the Internet on an old computer or just slow Internet connection. You can try it out yourself, just use the throttling feature in the developer tools for Firefox or Chrome/chromium. Now, browse the Internet like that for a couple of hours, try to do the stuff you regularly do. It will really make you appreciate so-called modern web development (sarcasm, just in case you didn't get that).
You need to select the
Network pane. Then check the
Disable caching option and choose a throttle speed. On Firefox, try the
Regular 2G speed. On Chrome/Chromium you can manually type in the download and upload speeds in kbit/s.
Some websites are so poorly built that you simply cannot use them.
Ingenious software is simple and simplicity is the heart of the Unix philosophy. The more code lines you have removed, the more progress you have made. As the number of lines of code in your software shrinks, the more skilled you have become and the less your software sucks. So you need to ask yourself: Do I really need that?
We used to sit around in the UNIX Room saying, "What can we throw out? Why is there this option?" It's often because there is some deficiency in the basic design — you didn't really hit the right design point. Instead of adding an option, think about what was forcing you to add that option.
― Doug McIlroy.
Modern programming scares me in many respects, where they will just build layer after layer after layer that does nothing except translate.
― Ken Thompson
A website is basically a document. Documents published on the Internet are in HTML, which is a standard markup language for documents designed to be displayed in a web browser. HTML has structure, e.g. "this is a title", "this is a link", and "this is a paragraph". HTML is responsive by default and it is very fast. Take a look at this basic HTML page and try to change the size of the view screen. It works everywhere on any size of screen.
Then you shake it up with some CSS, which is a style sheet language used for describing the presentation of a document written in a markup language such as HTML. CSS adds some shiny to the structure. CSS says, "let's make all the paragraphs have tiny text and brown colors" and "let's move this image to the middle of the screen and add a black border". And now everything suddenly looks amazing. Take a look at this basic HTML page with some CSS added.
Yes, I know. It's called messing up.
Input validation belongs on the server and you should never only do input validation in the client in the first place.
Since you need to do input validation on the server anyway, you might as well just do that. If done properly, you validate the entire form in one go and then the user will not notice any difference. And basic navigation, such as using the tabulator key to jump between fields, still work correctly then.
A web application is a website that has so much functionality that it works like a regular desktop application. Google Maps is a web application. GitHub is a web application. YouTube and Gmail are web applications.
"But we need a framework because we work in a team!"
"Are you saying we need to write everything ourselves from scratch?"
No, but just like if we were e.g. designing a new car, we would not begin by grabbing the toolbox, likewise proper web development does not begin by grabbing a framework or a library.
We need to strike a balance between performance, security, convenience, design and development principles.
Performance, security, convenience and principles in relation to software development
Performance, security, convenience, design and development principles are the issues we need to handle well in order to strike a sound balance.
Performance in relation to software is about getting the software to perform as efficiently as possible. It's primarily about speed and about fully utilizing the hardware resources we have available.
In e.g. game development, performance is important in order to avoid stuttering of video and audio. The less optimized a game is, the more CPU and GPU power it requires. If the game is poorly optimized it can only run on the most powerful equipment without stuttering, if at all. Power requirements in relation to wattage also increase many times.
In relation to web development, performance is important in order to avoid slow loading speeds, as we have seen. But loading speed is not the only concern. Hardware requirements are important considerations too. If a busy website isn't optimized, the hardware requirements for the web server can multiply by as much as 10 (or even more). This is not only more expensive, but it also has a big impact on the environment as much more electrical power is required to run multiple servers.
Security in relation to web development is a branch of Internet security which again is a branch of computer security. It covers a big area of concern such as proper identification and authentication, prevention of theft, prevention of denial of service, prevention of tampering and vandalism, and much more.
A lack of proper security regarding web development can affect both the server and the client.
Since the Internet is an inherently insecure channel for information exchange, with high risk of intrusion, insecure websites and web applications are often an entry point for attackers.
Convenience is about procedures, products and services intended to increase ease in accessibility and usage. It's about saving resources such as time, effort and energy and generally decrease frustration.
In relation to software and web development, convenience is about saving time and human effort.
Convenience is very important because it saves time, human energy and effort. However, as with everything else, we need a balance. Something can be too convenient and there are also such things as laziness, negligence and irresponsibility.
Generally a principle is a guide or rule for behavior or evaluation. In relation to design specifically, principles serve as guidelines for how to construct, design and build stuff. In relation to software development, whether web related or otherwise, principles concern both performance, security and convenience.
Some principles are based upon sound and practical experience, such as the principle of separation of concerns (SoC). Other principles, such as many of the different design patterns, have no real life benefit, on the contrary because they are mainly based upon theory, they often complicates the development of software unnecessarily.
Getting the balance right
Ideally we want everything to be as performant, as secure, and as convenient as possible and we want to be able to rely upon sound principles that help us achieve those goals.
A Formula One car is built for optimal performance, not for convenience, as such it is very uncomfortable to drive and it requires both a good physique and a strong mental discipline.
Is a Formula One car secure?
Well, not compared to a Knight XV SUV that has ballistic hardened steel and ballistic fiberglass fenders and bumpers making it fully armored.
But we don't need ballistic hardened steel, ballistic fiberglass fenders and bumpers on a Formula One car. Even though that would make it much more secure, it would also make it very slow and completely unsuited for racing.
Balance is achieved on a case by case evaluation of the specific requirements of the problem we're trying to solve. The less we understand the problem and the architecture, the less we're able to reach a good balance.
Since we're doing web development, we need to prioritize performance and security. Convenience is important too, but not if it compromises performance or security.
This doesn't mean that we do all web development in Rust in order to make the website or web application fast, but we should consider our approach. If we use an interpreted programming language we need to realize that every framework, every ORM library, and every template engine we put on top is another big layer of unneeded complex and slow code. The more custom made and discrete code we use, the faster the web application becomes.
Rather than using a framework, it's much better to use small libraries. Even better to just take out the specific code you need from the library, and then use that. Integrate it into the application as native code. This is another good way to make sure you really understand how the code works in your project.
The decision making process
Good software development, which involves web development, is a decision making process in which the basic sciences are applied in order to use resources optimally to meet stated objectives. As such, a fundamental element of this process is the establishment of objectives and criteria, analysis, construction, testing and evaluation.
Let's consider the bare minimum in the process.
- Determine the requirements
First we need to determine clearly what kind of problem we're trying to solve. Are we building a webshop or the next iteration of Google Maps?
- Analyze the problem
So we're building a webshop. What features does this webshop require and what challenges does it provide? Users potential geographical location is relevant. The importance of performance, security, and convenience must be determined in relation to all these issues.
- Plan or design the software-based solution
Determine what solution best solves the problem in optimal balance between performance, security, and convenience.
- Develop the software
- Test the software
- Deploy the software
- Maintain the software
Long term consequences are important considerations. The idea that we ship first and fix later rarely works and if you screw up, you cannot fix it later because then you will be out of business. This also means that you should avoid all the hyped-up tech, development methodologies and deployment methodologies that are promoted as "the modern approach" and then embrace the "boring". The boring represents that which has been battle tested and proven over the years to work really well.
The hype over some specific product or method will eventually settle and either be discarded as useless or join the ranks of the boring.
My advice is that you don't sign-up your project to be a guinea pig for hyperbole.
Since a lot of design principles only concern theoretical problems we need to get rid of all that too.
Two things help greatly in discarding useless principles:
- Pragmatic experience
If we have no experience, we need to stand on the shoulders of others. "I am building X and I am trying to achieve Y, will principle Z benefit me in relation to that? Why? How? What are the pros and cons?"
We need to search for people with real life experience and avoid code methodology cult members.
Investigation includes both study and a trial and error approach during testing, which eventually will lead to experience.
If you have a small application that only does a simple set of tasks, do you really need to consider e.g. the principle of separation of concern (SoC)? Does it make the maintainability of the software more convenient? Six months from now, will it help you understand your code better? Does it make the software run less efficient? Try it out and compare the results.
As software developers we need to be able to take at least a tiny amount of pride in the work we do and we need to try our best to make good solutions. Why should always strive to keep our dignity and not sell our souls.
I am just doing the job I am paid to do. If I don't do it, someone else will anyway.
― Typical excuse stated by people who compromise their integrity
We also need to have an engineering attitude towards technology. We need to be curious about HOW the technology works and WHY. It is important in order to use the right tools and use them correctly, we cannot do that if we don't understand the tech.
I know working as a web developer for a company or a client that doesn't care about proper software development can be very frustrating. I personally reject such jobs as I don't want to be a part of something like that, but I have previously had clients who turned out to be people who's only concern were the speed of the development process, I didn't initially know that, but then they starting cutting corners. This has long term devastating consequences for the business.
Good companies exist out there too, companies who really care. Some have learned the hard way. Others have just been blessed with the right people or the right attitude from the get go.
When we have a choice, we should look at the products the company make and then ask ourselves if that is something we want to be associated with.
Let's stop making rubbish and try to work to make the Internet a better place for all.
- Understanding the Digital World by Brian Kernighan.
- Coders at Work: Reflections on the Craft of Programming by Peter Seibel.