Oct 24, 2023·edited Oct 25, 2023Liked by N.S. Lyons
For several years I was a principal engineer at Amazon and I was a periodic member of the "availability team" which did post-mortem analysis of every event that caused the web site to go down. Amazon is/was the most operationally competent company I have ever worked for but the scale at which they operated required a level of complexity that no one could really get their arms around. No one fully understood the inter-relationships and dependencies between all the micro-services. The system required intense, continuous automated monitoring to achieve our goals in terms of uptime/availability. (At one point, fully 20% of the entire system's compute capacity was being consumed merely to process faux requests that checked on whether the system was up.) Everything you say about the challenges of complex systems is true. But I will offer a couple of other observations.
First, when faced with complex systems, human beings have a strong propensity to develop superstitious explanations for the visible phenomena they observe. (The last talk I gave to the engineering organization before leaving Amazon was called "Superstitious Architectures: How to Avoid Them".) Prying understanding from complex systems is hard and people are often happy to settle for superstition. (I offer as evidence a lot of what the lay press is saying about AI right now.) What this means in practice is that complexity causes human actors to operate in a state of greater general ignorance. Not a happy circumstance when the sh*t hits the fan.
Second, people are being driven to despair of their own human agency as their dependence on technology they can never understand grows. This is not entirely accidental I suspect and is probably even a little sinister. This has the effect of increasing dependency within the general population and reducing the sum total of initiative in the population at large.
Your advice to focus on simplifying our own lives is wise and even prescient I suspect.
The failure cascade of complex systems is not limited to war. Consider the collapse of Just in Time inventory systems during the COVID lockdowns. One bottleneck proved to be able to shutdown whole industries. A major disaster loomed when the Governor of Pennsylvania shut down all of the rest stops and restaurants along the PA interstates. This effectively cut the major supply line to NYC. President Trump and the President of the Teamsters "explained" things and got the order rescinded.
Going back to the defense of Israel, they had apparently forgotten the failure of the Bar Lev Line in 1973. Technology aside crust defenses are inherently fragile. See also Maginot Line and Great Wall of China. If you depend solely on a crust defense, once the enemy breaks the crust, they have a free run in the vacuum behind the crust. What is needed is defense in depth to contain the attack while larger forces mobilize to drive it back. Especially in an age of terrorism, an armed citizenry is indispensable. Citizens are the targets of terrorism so by definition they are the first to fight. Behind them are police, regional military (National Guard in an American context) and then the regular military.
Wonderful read. Insightful. Old wisdom, "the more moving parts, the more points of failure," applies to everything in life. Making anything more complex adds risk. Sometimes that complexity is really worth it; things are better with it than without it. But to not know, or deny, or ignore that old wisdom is foolish.
I laugh to myself, because so often reading your essays validates something I think already. And that's dangerous. Seductive. That feeling of "Yes! Exactly!" is just so good.
Oct 24, 2023·edited Oct 24, 2023Liked by N.S. Lyons
This feels like a timely critique of technology for technology's sake. It feels like humans are being encouraged to be less and less human, while being more and more entwined in technology that is unnecessary and not more useful than tried and true solutions from the past.
Hopefully more people will take a step back and push back against the wave of fragile tech that weakens us.
The towers being staffed exclusively by young women seems distinctly under-analyzed here. If we're serious about "people, ideas, machines - in that order" then the people aspect must have harsh questions be raised here.
Beyond the questionable wisdom of having an army with female soldiers to begin with, how is 100% of towers having female soldiers even demographically possible without extreme levels of wokeness and anti-male bias in staffing decisions? According to Wikipedia, in 2014 women made up only 4% of IDF combat roles with all the rest in support positions? Is it possible the IDF classified border posts as non-combat roles and used them as a form of demographic ballast to score virtue points? If so, why is nobody talking about that?
Strangely, the article left me in mostly in agreement with the solutions, though the analysis is poor. There are many assertions made about the fence as a system—this is nonlinear, emergent behavior, etc.—without citing any real sources about how it worked. The use of failure cascade here does not match my understanding either. A failure cascade is the proverbial house of cards collapsing as a result of a single point of failure. What Hamas executed was a systematic dismantling of a complex system, predicated on digital communication and surveillance. Anyone that's had to work a radio in wartime will tell you both how important communications are and how unreliable they are. The worst part was the example of EVs as being a less complex system. In fact, EVs are far more simple devices (see how much unions worry they'll lose jobs from having fewer parts to install). They do not have the benefit of a hundred years of refinement and innovation that internal combustion engines enjoy in both their design and production. So then, which is preferable? Simplicity or mature, proven technological systems?
Where this analysis succeeds is rooting the solutions in people. Boyd's principles are excellent and a great inclusion that illuminate the strategy for successfully deploying technology. I am curious if the IDF ever attempted a penetration test of their own fence, and if they came up with Hamas/Iran's plan. If not, why not? Similarly, they could have employed the "chaos monkey" approach, where you simply remove a component of your system and see how resilient it is in the face of one or multiple failures. Simply removing wireless communications would have highlighted how fragile this system was. Frankly, I would not be surprised to learn there was no stress-testing of such a system usually because they're politically unviable. If you invested political capital in making the fence, you're unlikely to want evidence from experts it doesn't work. The root failure mode, though, is a leadership failure and maybe a lack of imagination, neither of which can be solved with technology. My goal here is not to argue the fence is a good idea (evidence shows it's pretty terrible), but simply how future systems (including men with guns on a wall) should be evaluated and tested to gauge their merit.
There was no "border failure," nor was there a "technology" failure. This whole event was known about ahead of time and deliberately allowed to happen for strategic purposes. Nothing could be more obvious, and many Israelis are now speaking out.
There's a workplace politics aspect to explore here as well. Either the people responsible for the wall were too incompetent or too insecure to attempt red-team exercises against their designs.
If it's truly the case that disabling the cell networks effectively disables the guard towers and the gun bots, then I can't believe they ever made an attempt to defeat their own system. Redundancy here is cheap and critical. Hundred dollar handheld radios could have kept comms online between guard towers.
Static defense is usually vulnerable to dynamic, innovative, offense. Monash's synchronized offensive battle plans probably ended the First World War a year earlier than it would have otherwise. Synchronizing the offensive battlefield tactics of the new armaments, tanks, airplanes, artillery, in an engineered offensive with limited objectives decisively ended the stalemate. France's Maginot Line was pathetically ineffective in 1940 against the German Blitzkrieg.
It was the effectiveness of Hamas' cheap drones that foreshadow the future of war. There is nothing we can do to stop "everybody" from building lethal, autonomous, killing machines that operate without human control. Israel can build a wall of drones to surround Gaza that simply execute human beings who venture across the line. No paraglider lands alive. No human beings cross back alive. The killing machines will be as ubiquitous in battle as smartphones are in a queue. Why would we give advantage to the enemy and restrain our technology? Combatant? Non-combatant? That's a history lesson.
I'd like someone to refute my prophesy. Please, please, please, tell me I am wrong.
When Zionism was in its infancy, surveyors were sent to look over the proposed new land and report back their findings about the suitability of the territory for settlement and agriculture. They reported back that almost all the arable land had Arab farmers living on it. This information was ignored.
This policy, of ignoring the Palestinian problem, ran into its first violent failure with the 1928 riots and massacre of Jews in Hebron, followed by the Arab revolt in 1936 (crushed by the British as a colonial war) the 1948 war, with the expulsion of 750,000 Arabs and the creation of the refugee problem, then the ’67 war and the creation of the occupied West Bank, and so on and so on.
By the time of the October 7 massacre of Israeli towns along the Gaza border, Israeli’s were living with this prolonged denial of the existence of a Palestinian problem, by protecting itself with a barrier wall separating Israel proper from the West Bank, the Iron Dome and the Gaza security fence. They were able to convince themselves that they were safe from any Palestinian violence because the Palestinians were walled away, out of sight, out of mind. Demands for an actual solution to the problem, Oslo, two state solution, whatever, faded away. Like the early Zionists, they ignored the problem.
It was technology that abetted this denial, but in the end it was the human tendency to avoid the difficult, to look for easy solutions, to wish away the distasteful. The result was October 7th.
It comes as a relief to read a 'right-wing' thinker who does not default to conspiracism.
The instant 'inside job' theories again reminded me that it's psychologically comforting to believe that wired-in human frailty isn't behind intelligence failures. The entirely normal human tendency to err and fall into hubris only underlines our vulnerability. That is a much more frightening forever threat than evil plotters getting together to do something bad every now and then.
Thought-provoking essay, as always. I hadn't yet bothered to learn about the tactics of the Hamas attack (maybe because, as an American, I assume Southern borders are naturally porous---but that's a separate issue), so this was an enlightening summary of what went down on 10/7. I've nothing to add, except to re-emphasize one aspect of the attack which you touched on twice:
"Hamas, however, used small, off-the-shelf drones rigged with mortar rounds and other explosives to attack and disable the communications towers powering the network. These drones were too small and low-flying for radar to detect, so would have had to have been spotted by eye and ear."
The WSJ article you linked to is behind a paywall, but I assume it describes the same phenomenon that observers of the Russo-Ukrainian war have noted over the past year and a half: these small drones are lethal. They constitute a revolution in military affairs. The Russkies use them for reconnaissance / surveillance tasks, as well as generating telemetry data for artillery strikes, naturally, but will also pilot them, silently, over a Ukrainian trench, then drop grenades onto its occupants. This is terrifying for infantry, and unimaginable before this era. Not to mention Russia's medium-size Lancet kamikaze drones, which casually conduct precision strikes against tanks and armored troop carriers. It used to be the case if, from the air, you wanted to take out one of these, it required a burst from an A-10 Warthog's auto-cannon or an expensive air-to-ground missile. No longer.
So here we are, back at a new drawing board. Think what you will about the villain of A Few Good Men, Colonel Jessup, but it's difficult to discount his supremely realist perspective on such matters:
"You can't handle the truth! Son, we live in a world that has walls, and those walls have to be guarded by men with guns. Who's gonna do it? You[, Lieutenant Kaffee]? You, Lieutenant Weinberg?* I have a greater responsibility than you can possibly fathom. You weep for Santiago and you curse the Marines. You have that luxury. You have the luxury of not knowing what I know: Santiago's death, while tragic, probably saved lives. And my existence, while grotesque and incomprehensible to you, saves lives. You don't want the truth because deep down in places you don't talk about at parties, you want me on that wall---you need me on that wall."
The Collapse of Complex Societies by Joseph Tainter makes similar points. He’s an archaeologist but his analysis isn’t limited only to those complex societies that have left only physical evidence but few records.
I don't know if you have seen this - very interesting, fact-based explanation implies the move to one source of software was big part of the problem, rather than having multiple systems and backups.
For several years I was a principal engineer at Amazon and I was a periodic member of the "availability team" which did post-mortem analysis of every event that caused the web site to go down. Amazon is/was the most operationally competent company I have ever worked for but the scale at which they operated required a level of complexity that no one could really get their arms around. No one fully understood the inter-relationships and dependencies between all the micro-services. The system required intense, continuous automated monitoring to achieve our goals in terms of uptime/availability. (At one point, fully 20% of the entire system's compute capacity was being consumed merely to process faux requests that checked on whether the system was up.) Everything you say about the challenges of complex systems is true. But I will offer a couple of other observations.
First, when faced with complex systems, human beings have a strong propensity to develop superstitious explanations for the visible phenomena they observe. (The last talk I gave to the engineering organization before leaving Amazon was called "Superstitious Architectures: How to Avoid Them".) Prying understanding from complex systems is hard and people are often happy to settle for superstition. (I offer as evidence a lot of what the lay press is saying about AI right now.) What this means in practice is that complexity causes human actors to operate in a state of greater general ignorance. Not a happy circumstance when the sh*t hits the fan.
Second, people are being driven to despair of their own human agency as their dependence on technology they can never understand grows. This is not entirely accidental I suspect and is probably even a little sinister. This has the effect of increasing dependency within the general population and reducing the sum total of initiative in the population at large.
Your advice to focus on simplifying our own lives is wise and even prescient I suspect.
Thanks for this essay.
Excellent essay.
The failure cascade of complex systems is not limited to war. Consider the collapse of Just in Time inventory systems during the COVID lockdowns. One bottleneck proved to be able to shutdown whole industries. A major disaster loomed when the Governor of Pennsylvania shut down all of the rest stops and restaurants along the PA interstates. This effectively cut the major supply line to NYC. President Trump and the President of the Teamsters "explained" things and got the order rescinded.
Going back to the defense of Israel, they had apparently forgotten the failure of the Bar Lev Line in 1973. Technology aside crust defenses are inherently fragile. See also Maginot Line and Great Wall of China. If you depend solely on a crust defense, once the enemy breaks the crust, they have a free run in the vacuum behind the crust. What is needed is defense in depth to contain the attack while larger forces mobilize to drive it back. Especially in an age of terrorism, an armed citizenry is indispensable. Citizens are the targets of terrorism so by definition they are the first to fight. Behind them are police, regional military (National Guard in an American context) and then the regular military.
Wonderful read. Insightful. Old wisdom, "the more moving parts, the more points of failure," applies to everything in life. Making anything more complex adds risk. Sometimes that complexity is really worth it; things are better with it than without it. But to not know, or deny, or ignore that old wisdom is foolish.
I laugh to myself, because so often reading your essays validates something I think already. And that's dangerous. Seductive. That feeling of "Yes! Exactly!" is just so good.
This feels like a timely critique of technology for technology's sake. It feels like humans are being encouraged to be less and less human, while being more and more entwined in technology that is unnecessary and not more useful than tried and true solutions from the past.
Hopefully more people will take a step back and push back against the wave of fragile tech that weakens us.
The towers being staffed exclusively by young women seems distinctly under-analyzed here. If we're serious about "people, ideas, machines - in that order" then the people aspect must have harsh questions be raised here.
Beyond the questionable wisdom of having an army with female soldiers to begin with, how is 100% of towers having female soldiers even demographically possible without extreme levels of wokeness and anti-male bias in staffing decisions? According to Wikipedia, in 2014 women made up only 4% of IDF combat roles with all the rest in support positions? Is it possible the IDF classified border posts as non-combat roles and used them as a form of demographic ballast to score virtue points? If so, why is nobody talking about that?
Strangely, the article left me in mostly in agreement with the solutions, though the analysis is poor. There are many assertions made about the fence as a system—this is nonlinear, emergent behavior, etc.—without citing any real sources about how it worked. The use of failure cascade here does not match my understanding either. A failure cascade is the proverbial house of cards collapsing as a result of a single point of failure. What Hamas executed was a systematic dismantling of a complex system, predicated on digital communication and surveillance. Anyone that's had to work a radio in wartime will tell you both how important communications are and how unreliable they are. The worst part was the example of EVs as being a less complex system. In fact, EVs are far more simple devices (see how much unions worry they'll lose jobs from having fewer parts to install). They do not have the benefit of a hundred years of refinement and innovation that internal combustion engines enjoy in both their design and production. So then, which is preferable? Simplicity or mature, proven technological systems?
Where this analysis succeeds is rooting the solutions in people. Boyd's principles are excellent and a great inclusion that illuminate the strategy for successfully deploying technology. I am curious if the IDF ever attempted a penetration test of their own fence, and if they came up with Hamas/Iran's plan. If not, why not? Similarly, they could have employed the "chaos monkey" approach, where you simply remove a component of your system and see how resilient it is in the face of one or multiple failures. Simply removing wireless communications would have highlighted how fragile this system was. Frankly, I would not be surprised to learn there was no stress-testing of such a system usually because they're politically unviable. If you invested political capital in making the fence, you're unlikely to want evidence from experts it doesn't work. The root failure mode, though, is a leadership failure and maybe a lack of imagination, neither of which can be solved with technology. My goal here is not to argue the fence is a good idea (evidence shows it's pretty terrible), but simply how future systems (including men with guns on a wall) should be evaluated and tested to gauge their merit.
There was no "border failure," nor was there a "technology" failure. This whole event was known about ahead of time and deliberately allowed to happen for strategic purposes. Nothing could be more obvious, and many Israelis are now speaking out.
There's a workplace politics aspect to explore here as well. Either the people responsible for the wall were too incompetent or too insecure to attempt red-team exercises against their designs.
If it's truly the case that disabling the cell networks effectively disables the guard towers and the gun bots, then I can't believe they ever made an attempt to defeat their own system. Redundancy here is cheap and critical. Hundred dollar handheld radios could have kept comms online between guard towers.
Static defense is usually vulnerable to dynamic, innovative, offense. Monash's synchronized offensive battle plans probably ended the First World War a year earlier than it would have otherwise. Synchronizing the offensive battlefield tactics of the new armaments, tanks, airplanes, artillery, in an engineered offensive with limited objectives decisively ended the stalemate. France's Maginot Line was pathetically ineffective in 1940 against the German Blitzkrieg.
It was the effectiveness of Hamas' cheap drones that foreshadow the future of war. There is nothing we can do to stop "everybody" from building lethal, autonomous, killing machines that operate without human control. Israel can build a wall of drones to surround Gaza that simply execute human beings who venture across the line. No paraglider lands alive. No human beings cross back alive. The killing machines will be as ubiquitous in battle as smartphones are in a queue. Why would we give advantage to the enemy and restrain our technology? Combatant? Non-combatant? That's a history lesson.
I'd like someone to refute my prophesy. Please, please, please, tell me I am wrong.
This is a great complement to Harold Robertson's recent speculations about another source of failure within our complex systems--namely, the purported decline in competence-based hiring and promotion: https://www.palladiummag.com/2023/06/01/complex-systems-wont-survive-the-competence-crisis/
Tech won't save us from ourselves.
When Zionism was in its infancy, surveyors were sent to look over the proposed new land and report back their findings about the suitability of the territory for settlement and agriculture. They reported back that almost all the arable land had Arab farmers living on it. This information was ignored.
This policy, of ignoring the Palestinian problem, ran into its first violent failure with the 1928 riots and massacre of Jews in Hebron, followed by the Arab revolt in 1936 (crushed by the British as a colonial war) the 1948 war, with the expulsion of 750,000 Arabs and the creation of the refugee problem, then the ’67 war and the creation of the occupied West Bank, and so on and so on.
By the time of the October 7 massacre of Israeli towns along the Gaza border, Israeli’s were living with this prolonged denial of the existence of a Palestinian problem, by protecting itself with a barrier wall separating Israel proper from the West Bank, the Iron Dome and the Gaza security fence. They were able to convince themselves that they were safe from any Palestinian violence because the Palestinians were walled away, out of sight, out of mind. Demands for an actual solution to the problem, Oslo, two state solution, whatever, faded away. Like the early Zionists, they ignored the problem.
It was technology that abetted this denial, but in the end it was the human tendency to avoid the difficult, to look for easy solutions, to wish away the distasteful. The result was October 7th.
It comes as a relief to read a 'right-wing' thinker who does not default to conspiracism.
The instant 'inside job' theories again reminded me that it's psychologically comforting to believe that wired-in human frailty isn't behind intelligence failures. The entirely normal human tendency to err and fall into hubris only underlines our vulnerability. That is a much more frightening forever threat than evil plotters getting together to do something bad every now and then.
Thought-provoking essay, as always. I hadn't yet bothered to learn about the tactics of the Hamas attack (maybe because, as an American, I assume Southern borders are naturally porous---but that's a separate issue), so this was an enlightening summary of what went down on 10/7. I've nothing to add, except to re-emphasize one aspect of the attack which you touched on twice:
"Hamas, however, used small, off-the-shelf drones rigged with mortar rounds and other explosives to attack and disable the communications towers powering the network. These drones were too small and low-flying for radar to detect, so would have had to have been spotted by eye and ear."
The WSJ article you linked to is behind a paywall, but I assume it describes the same phenomenon that observers of the Russo-Ukrainian war have noted over the past year and a half: these small drones are lethal. They constitute a revolution in military affairs. The Russkies use them for reconnaissance / surveillance tasks, as well as generating telemetry data for artillery strikes, naturally, but will also pilot them, silently, over a Ukrainian trench, then drop grenades onto its occupants. This is terrifying for infantry, and unimaginable before this era. Not to mention Russia's medium-size Lancet kamikaze drones, which casually conduct precision strikes against tanks and armored troop carriers. It used to be the case if, from the air, you wanted to take out one of these, it required a burst from an A-10 Warthog's auto-cannon or an expensive air-to-ground missile. No longer.
So here we are, back at a new drawing board. Think what you will about the villain of A Few Good Men, Colonel Jessup, but it's difficult to discount his supremely realist perspective on such matters:
"You can't handle the truth! Son, we live in a world that has walls, and those walls have to be guarded by men with guns. Who's gonna do it? You[, Lieutenant Kaffee]? You, Lieutenant Weinberg?* I have a greater responsibility than you can possibly fathom. You weep for Santiago and you curse the Marines. You have that luxury. You have the luxury of not knowing what I know: Santiago's death, while tragic, probably saved lives. And my existence, while grotesque and incomprehensible to you, saves lives. You don't want the truth because deep down in places you don't talk about at parties, you want me on that wall---you need me on that wall."
*Apropos
The Collapse of Complex Societies by Joseph Tainter makes similar points. He’s an archaeologist but his analysis isn’t limited only to those complex societies that have left only physical evidence but few records.
You did good by not rushing to come out with a take. I assume you're familiar with https://tinkzorg.wordpress.com/2021/10/20/welcome-to-jurassic-park/?
I don't know if you have seen this - very interesting, fact-based explanation implies the move to one source of software was big part of the problem, rather than having multiple systems and backups.
https://twitter.com/PatrickByrne/status/1712536185065844775?s=20