There was a massive Verizon outage on January 14. Reuters reported that it lasted 10 hours. Downdetector said it received 2.2 million reports of problems with Verizon’s service, but other estimates are as low as 180,000. As of this writing, Verizon hasn’t announced a reason for the outage. Cybersecurity concerns are possible, but have been mostly ruled out. It appears to have been an internal “technical issue.”

I can’t tell you what the specific technical issue was, but I can tell you what the general issue was: massive centralization.

Companies design systems with massive geographic centralization for cost and convenience, not for resilience – or, for that matter – cybersecurity.

Information and communications industries as a whole have been moving steadily towards massive centralization for several years now. Decentralize command and control. Centralization is a military-grade problem. There will be bigger and more impactful outages across all industries while we re-learn this lesson.

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #NetworkArchitecture #SystemDesign

Winter weather – power failures – what’s a good design?

In theory – in a perfect world – the backup batteries only need to last long enough for the generator(s) to start up and stabilize with the load of your choice. But, in the real world, the backup batteries should hold the system up in the following scenario:

1. Power goes out.
2. Batteries/UPS take the load.
3. Power failure alarm is issued to the technician on call.
4. Generator fails to start.
5. Generator failure alarm is issued to the technician on call.
6. The tech on call requests service from the generator maintenance contract company.
7. The generator company rolls a truck.
8. The generator service person identifies the problem, repairs it, and starts the generator.

If continuous operation through a power failure is the goal, I design battery/UPS systems for a minimum six hours of run time, and if the generator company has to roll a truck that’s really not enough. Six hours is only enough if you have in-house technicians on call who live close to the monitored system.

If it’s impractical to support a system with the appropriate amount of battery capacity for a generator repair, then the solution is a second generator. If, and only if, the system is protected with a second generator, is it feasible to reduce the battery capacity. Keep in mind that battery capacity decreases over the life of the battery, or with temperature variations, etc. Also, equipment gets added over time, so if the system is built with marginal capacity (generator startup and RPM stabilization), then when you have an outage six months or a year after initial installation, the batteries may no longer be adequate.

Design with lots of margin, not just to load transfer time.

#CallMeIfYouNeedMe #FIFONetworks

#resilience #NetworkArchitecture #infrastructure

A few months ago I discovered a law firm’s financial information (specifically billing and payment information), online. It’s a nationally known law firm, and the records in question were for the Seattle office.

Broken down by customer.
Itemized hourly billing.
Hourly billing rate.
Other expenses.
Customer account number.
Customer payment information, including bank account number.
Law firm’s bank account number.
Amounts paid.
Payment dates.
Balance due.
The information did NOT include details of the services provided.

I found it entirely by accident, with a Google search that wasn’t targeted in nature.

No, I didn’t report it to the law firm. In Washington, “Good faith acquisition of personal information . . . is not a breach of the security of the system when the personal information is not used or subject to further unauthorized disclosure.” (RCW 19.255.005(1))
I believe that protects me, but I don’t want to test it in court, and if the law firm knew about it, they might feel compelled to take some sort of action other than securing their information better.

THE LESSON
Do not store your company records, and host your website, on the same server. I can’t believe I have to write that sentence.

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #NetworkArchitecture #SystemsDesign #policy

Concerned about AI-generated malware bringing down your company? Then get your critical data off the Internet.

This isn't rocket science. This is Occam’s Razor.

The Internet is for social media and retail sales.

PII, PHI, employee records, customer information - nothing important should ever be Internet accessible.

Ever heard of private data circuits? Private data circuits are a real thing. People quit using them because the Internet was cheaper.

"It'll be secure," they said.

No. The Internet has never been secure. The Internet cannot ever be secure, because authenticated users will always be tricked into doing stuff for cybercriminals.

If there was ever a time to rethink your business strategy as it relates to information storage and processing, that time is now.

It's going to get worse quickly. Your best defense is to get sensitive data out of the public cloud.

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #NetworkArchitecture #SystemsDesign #policy

“Basically it's a 1995 AOL chat room and you have, like, a wheel-speed sensor that's shouting AGE/SEX/LOCATION over and over in group chat.

Now you can understand how your corroded backup camera will strand you.” #car #networkarchitecture https://zeroes.ca/@subjacentish/115432294616786461

A lot of the work I do is in high security systems where sensitive data isn’t connected to the Internet, and isn’t hosted on commercial public cloud platforms, because such an architecture can’t meet the design criteria.

A recurring issue I face is educating new decision makers who get ill-informed notions that they can reduce costs (thereby becoming heroes, or so they think), by centralizing information storage or processing on rented commercial platforms. So I go through it all again, patiently, politely, with the new person.

The other recurring threat I deal with is C-level people who want what I refer to as Data Ubiquity: “I want access to all of the data, at any time, from any location, on any of my devices.”

Data Ubiquity = Maximum Vulnerability.

Even “perfect” authentication won’t prevent this vulnerability. Why? Phishing. The authenticated user will be tricked into opening the door for the cybercriminal.

When the data is in no way Internet connected, how does the victim deliver the data to the cybercriminal? Do they print it out and ship reams of paper in boxes to the criminal via FedEx?

Offline Data = More Secure Data.

The cloud is for retail sales and social media, NOT for PHI, PII, corporate secrets, intellectual property, employee records, industrial controls...

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #NetworkArchitecture #policy #privacy

🚀 Wow, a riveting introduction to *yet another* #protocol no one asked for! 🤯 Bluesky's #AT Protocol explained for developers who want to understand network architecture instead of, you know, actually getting work done. 🎉 Because what we all need is more jargon to spice up our lives! 😂
https://mackuba.eu/2025/08/20/introduction-to-atproto/ #Bluesky #networkarchitecture #developerjargon #technews #HackerNews #ngated

Level up your campus network! 🎓 Did you know MPLS is now a game-changer for SONiC-based switches?
Asterfusion is bringing the power of MPLS to campus networks, enabling:

Simplified L3 backbone architecture
Large-scale VPN isolation
Efficient campus-to-site interconnections

Our latest blog breaks down why MPLS is crucial and how it works with SONiC to deliver a more efficient and secure network.
Don't get left behind! Learn how to transform your campus network with this powerful combination. 👇

🔗 https://cloudswit.ch/blogs/sonic-based-campus-networks-with-mpls/

#MPLS #SONiC #CampusNetwork #NetworkArchitecture #Asterfusion #VPN #NetworkSwitch #ITInfrastructure

The routing world is about to change! 🌐 Asterfusion is taking SONiC beyond switches and into the router era with the powerful Marvell OCTEON 10 DPU!

For too long, the router market has been dominated by closed, legacy systems. But Asterfusion is changing the game with a true commercial-ready, open-source solution.

Our new ET Series Routers/Gateways are powered by SONiC-VPP, offering a high-performance, energy-efficient solution for:

Enterprise network egress
Service provider edge deployments

Ready to embrace the future of open networking? Click to see how Asterfusion is building the next generation of SONiC-based white-box routers! 👇

🔗 https://cloudswit.ch/blogs/sonic-router-era-with-marvell-octeon-10-dpu/

#SONiC #Router #NetworkArchitecture #Marvell #DPU #OpenNetworking #EdgeComputing #Asterfusion #WhiteBoxRouter #CloudComputing

Struggling to choose the right 32x100G switch for your AI or HPC network? 🤔 Stop guessing and start making an informed decision! Asterfusion's latest guide is here to help.

We break down the key differences between two powerful platforms: the Marvell Falcon and Teralynx 7.

Teralynx 7 is perfect for:
Ultra-low latency AI/ML training
High-Performance Computing (HPC)
Financial trading platforms

Marvell Falcon is ideal for:
Hyperscale cloud data centers
Multi-tenant environments
VXLAN-heavy overlay networks

Regardless of your choice, you receive our unified, production-ready Enterprise SONiC software, featuring virtualization, zero packet loss, and powerful automation capabilities.

Ready to make the smartest network investment? 👇

🔗https://cloudswit.ch/blogs/choosing-the-right-32x100g-switch/

#Switch #DataCenter #AINetwork #HPC #NetworkArchitecture #NetworkHardware #Asterfusion #SONiC #Marvell #CloudComputing #NetworkManagement

Launching the network into the sun is the strategic response. For a short term operational fix, should they just set fire to the desks and racks? Minimal capex for that, right?

#networkarchitecture

Over the weekend I set up an air-gapped computer for use with certain clients. The increasing use of Artificial Intelligence (AI) to analyze data of all types warrants this new operational procedure for my clients with Non-Disclosure Agreements (NDAs).

Examples of privacy violations are too numerous to count. To give you one example (that doesn’t even use AI), companies have been found guilty of violating user preferences regarding location tracking. Another example: so-called anonymized data has been connected back to the associated sources many times through the use of many methods. The analysis of anonymized data with AI tools makes it even easier to de-anonymize information.

Major software companies, operating system companies, device manufacturers, and cloud service providers are all actively working to obtain your data.

Legal protections are lagging behind technology advances.

Privacy policies are written to confuse. They deliberately include doublespeak and ambiguity.

Default opt-in is normalized.

AI systems are leaky. They have information they obtain without your informed consent, and they leak that information in ways the system owners can’t even predict.

You cannot avoid working with AI-enabled networks, hardware, software, and systems. Even when you try to minimize it, disable it, or reject it, your information is at risk.

For these reasons, I’m applying the following operational policies for information from any company for which I’ve signed an NDA:

1) I’m making available file transfer systems that are end-to-end encrypted. The use of these systems is at the client’s option. If they want to send a document as an unencrypted email attachment, they can still do that. I’ll support, and work with, any encryption methods the client chooses.

2) All information received under an NDA will be moved to the air-gapped system for processing. Even if they send me a document as an unencrypted PDF, I won’t open it with any application until it’s on the air-gapped system.

These steps don’t protect the client from all risks, but they do allow me to prove due diligence in protecting information provided to FIFO Networks under an NDA.

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #privacy #NDA #NetworkArchitecture #policy

When you set up your e-commerce platform on a Cloud Services Provider (CSP), be sure you understand how their load balancing works. Understand the geographic distribution of the load balancing system in relation to your e-commerce servers. Where are your customers “entering” the network, and where is their data transported for sales and payment events? Load balancing can significantly affect pricing, so you’ll want to engineer accordingly, and monitor continually.

#CallMeIfYouNeedMe #FIFONetworks

#NetworkArchitecture

Using the company website as the launch point for the employee login is a common practice. With adequate Identity and Access Management (IAM), it seems secure enough. But, there’s another piece to this.

When the well-known domain is the launch point for the employee login, it sometimes means that the employee data is stored on, and accessible from, the same server group, and in the same IP address range. In other words, the employee data may be accessible and downloadable without an employee’s authentication credentials.

I know of a law firm that has its billing and financial data literally on the same hard disk as their website. If the cybercriminal breaches the website, they have access to everything.

THE LESSON
The more separation you have between your public website and your private data, the better.

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #NetworkArchitecture

⚙️ Inside Asterfusion’s 800G AI Switch Architecture

AI workloads demand ultra-high bandwidth and low latency. Asterfusion’s 800G switch architecture delivers both—with a focus on open hardware, precision timing, and energy-efficient design.

In this post, we cover:
🔸 Why 800G switching is critical for next-gen AI clusters
🔸 How Asterfusion’s design balances speed, scale, and sustainability
🔸 The use of Broadcom Jericho3-AI & Tomahawk5 chips
🔸 Why their disaggregated model matters for AI networking

🔗 Explore the full breakdown: https://cloudswit.ch/blogs/asterfusions-800g-ai-switch-architecture

#800G #AISwitch #DataCenterNetworking #Asterfusion #Broadcom #OpenNetworking #HyperscaleAI #NetworkArchitecture

If you prefer centralization to segmentation,
If you prefer “cloud only” to cloud as a last resort,
If you prefer outsourcing your help desk to building tribal knowledge,
I’m not your huckleberry.

Why? Because...
I will tear your team apart.
I will deconstruct your team
and reconstruct your team.

As quickly as I can, I’ll replace the “cloud only” people with
people who value security over convenience, with
people who value security over lowest cost, with
people who value security over business as usual.

From Accounting to Marketing to Sales to Customer Care, every department will be involved in reviewing their part of the company’s data to determine what must be online, and what can be stored locally.

You will have a server room of your very own.
Your own IT personnel will hear the whirring of the servers’ fans.
I will replace some VPNs with more expensive dedicated circuits.
Some data will only be accessible by coming into the office.
In Customer Care, no single login will be able to access all customers’ sensitive information. Depending on the size of the company and call center, one Customer Care representative may only be able to access A-M, or A-G, or maybe even just A. If a cybercriminal phishes their login, that’s all the cybercriminal will get.

Everything will be backed up locally.
Everything.
Even what’s left in the cloud.
Yes, there will still be some data left in the cloud, like your store, where you sell stuff.
And there may still be some data off premise, in private cloud storage, on servers the company owns, rather than servers the company leases.
But, for the most part, the cybercriminal will have to enter the building to access the data.
Instead of having only one security mechanism (authentication), you will have two: authentication security plus location security.

If you can’t make a profit with this security model, one of two things are true.
Either
Your business model isn’t viable,
Or
You suck at running a business.

It has always been possible to run a profitable business without creating a global attack surface for your sensitive data.

The Cloud Sales Machine has done an incredibly effective job of convincing you
that if it’s not secure, it’s your fault,
that authentication is enough,
that cheap is just as good.

The Cloud Sales Machine has done a really, truly, amazingly, incredibly effective job with that last one: cheap.
It is rational for you to question whether your monthly subscription and service fees have gotten out of hand.
It is rational for you to be dismayed at the complexity of the pricing scheme, because it really is a scheme, carefully designed to hide the true cost in a swirling fog of mystery.

It is rational for you to think, “Maybe we could actually save money by pulling this in-house.”

But don’t lose sight of the objective.
The objective isn’t to do it the least expensive way possible.
The objective is to do it in a way that is secure, and still profitable.

If any of this makes sense to you,
I’m your huckleberry.

#CallMeIfYouNeedMe #FIFONetworks #cybersecurity #networkarchitecture

Someone recently asked me about the difference between network segmentation and data segmentation after I mentioned them in a post. Both are important. Sometimes you use one method, sometimes the other, and sometimes both. And then, karma. A perfect example of data segmentation appeared on my screen a day or two later, and now I’ll share it with you.

Here’s an example of data segmentation, possibly without network segmentation. See accompanying picture.

I have multiple websites with the same hosting company. The hosting company is offering me the option of merging all of my websites under one login. That would be convenient, but it’s less secure.

At the data level, a cybercriminal must authenticate on each of the websites separately, with separate 2FA. At the network level, I have no way of knowing if the web hosting company has segmented the infrastructure, and to what degree. For some companies, detailed knowledge of the hosting company’s physical architecture is essential to good security, but for me it doesn’t matter, since I have zero confidential information stored on, or accessible from, the web servers. The worst thing a cybercriminal can do to my websites is defacement or knocking them offline.

THE LESSON
As part of your risk assessment, consider both network segmentation and data segmentation. Everything that can be accessed from the same authentication credentials is in the same data segment. The most common weakness I uncover is in granting a single Administrator account too much access.

#CallMeIfYouNeedMe #FIFONetworks

#cybersecurity #NetworkArchitecture #DataArchitecture

https://www.linkedin.com/pulse/ai-data-center-evolution-juniper-apstras-lifecycle-solution-poller-uckle

#ArtificialIntelligence #DataCenter #NetworkAutomation #IntentBasedNetworking #JuniperNetworks #AIIFD2 #TechFieldDay #NetworkArchitecture #ITInfrastructure #AIInfrastructure
6/6

I just posted this controversial post on LinkedIn. Let the blocking begin. (It might happen here, too).

“Hi Bob, I hope that you're well! Do you do migrations? I’m looking for someone who can help me with migrating my <number redacted> employee company from <Cloud Provider Name Redacted> to <Cloud Provider Name Redacted> and help us avoid any gotchas, as we will still be hosting our production site on <Cloud Provider Name Redacted> for the time being. I'm realizing that paying an expert in this case will save time and opportunity. Thanks much!”

“<Name redacted>, thank you so much for thinking of me. In this instance, I’m not the right person for the job. I’d be the one to call if you wanted to migrate out of a commercial cloud service and maintain your operations in your own server room, or on your (owned) equipment in a private data center. You need someone with good knowledge of <Cloud Provider Name Redacted> cloud products. I studiously avoid an architecture that I consider fundamentally flawed.”

If the current chaos in the USA has caused you to realize that it’s important to keep local control of your IT operations, I’m the Network Architect/Engineer who can help you with that.

#CallMeIfYouNeedMe #FIFONetworks

#ProjectManagement #NetworkArchitecture #NetworkEngineering #Policy #CyberSecurity #InformationSecurity

A new Director asked me for help to quickly get a grasp of the network he was responsible for. I signed an NDA as part of the preparation for the engagement. Then the company sent me a purchase order, and we were ready to begin.

As we talked, I learned that he was the only in-house, on-the-payroll IT person. It was a hybrid system, both cloud and on-prem. The company was using a Managed Services Provider (MSP), and they provided two full-time, on site personnel. They did Help Desk support and any on-prem work that needed to be done in the server room.

“Okay,” I said, “ask them to give you a copy of the network diagram and send it to me. Then we’ll go over it together, and I’m sure I’ll have more questions, too.”

The two guys from the MSP couldn’t provide him with a network diagram. This MSP had been providing operations support for a couple of years. Both of the gentlemen from the MSP had been working on site for several months.

Soon, I was in a video conference with the Director and the two techs from the MSP. They didn’t like me much. I was always polite and respectful, but I kept asking questions they couldn’t answer. Someone at the MSP figured out that the account was in jeopardy and told the on-prem guys to cooperate with me. Things went much smoother after that.

THE LESSON
If you’re in IT upper management, ask your team for a network diagram. Even if the majority of your system is cloud-based, you’ve still got an Internet connection into the building, a firewall, and a distribution system. They should be able to provide this information instantly, and if they can’t, there are far bigger problems waiting to be discovered.

#CallMeIfYouNeedMe #FIFONetworks

#NetworkArchitecture #cybersecurity