Note: The job is a remote job and is open to candidates in USA. Marathon TS is seeking an experienced Senior Palo Alto Network Security Engineer to lead a production firewall remediation and operating system upgrade for a mission-critical enterprise environment. The role involves performing a controlled, zero-downtime upgrade of a High Availability Palo Alto firewall environment while addressing authentication issues affecting users.
Responsibilities
- Assess the current Palo Alto firewall environment and overall health
- Validate High Availability (HA) configuration, synchronization, and failover readiness
- Inventory PAN-OS versions, Dynamic Updates, GlobalProtect, plugins, and content releases
- Export and secure running configurations, device state files, and technical support files
- Evaluate Panorama dependencies and upgrade sequencing
- Establish baseline performance metrics including:
- CPU utilization
- Session counts
- Throughput
- VPN utilization
- Packet buffers
- Review licensing and support entitlement status
- Document upgrade readiness and identify technical risks
- Troubleshoot and resolve SAML/Single Sign-On authentication issues
- Validate synchronization between firewalls and Identity Providers (IdPs)
- Review and update:
- IdP metadata
- Signing certificates
- Certificate chains
- Entity IDs
- Assertion Consumer Service (Client) URLs
- Group mappings
- Authentication profiles
- Analyze authentication and system logs to isolate root causes
- Validate GlobalProtect authentication workflows
- Configure and verify local break-glass administrative access
- Develop a production-ready upgrade strategy
- Determine the appropriate PAN-OS target release
- Validate software compatibility with:
- Panorama
- GlobalProtect
- User-ID
- Dynamic Updates
- Plugins
- Routing services
- Stage software images
- Develop rollback procedures and recovery plans
- Produce detailed maintenance window documentation
- Define Go/No-Go decision points
- Execute a controlled rolling upgrade of a High Availability firewall pair while maintaining uninterrupted production services
- Validate HA health
- Upgrade passive firewall
- Verify synchronization and health
- Execute controlled failover
- Monitor:
- Routing
- VPN connectivity
- NAT
- Production traffic
- Session stability
- Upgrade remaining firewall
- Restore preferred HA state
- Validate overall operational health
- Validate administrative SSO functionality
- Validate GlobalProtect authentication
- Test role mappings and group-based access
- Verify HA failover behavior following upgrade
- Compare post-upgrade performance against baseline metrics
- Document metadata renewal procedures
- Produce final operational acceptance documentation
Skills
- Minimum 8 years of enterprise network security engineering experience
- Minimum 5 years administering Palo Alto Networks next-generation firewalls
- Demonstrated experience performing production PAN-OS upgrades in High Availability environments
- Strong understanding of Active/Passive HA
- Strong understanding of Active/Active HA
- Strong understanding of Session synchronization
- Strong understanding of Failover operations
- Extensive experience with GlobalProtect
- Extensive experience with User-ID
- Extensive experience with Dynamic Updates
- Extensive experience with Panorama
- Experience troubleshooting enterprise SAML authentication
- Strong knowledge of SAML 2.0
- Strong knowledge of Identity Providers (Microsoft Entra ID/Azure AD, Okta, Ping, ADFS, etc.)
- Strong knowledge of Certificates
- Strong knowledge of PKI
- Strong knowledge of NTP
- Experience developing rollback strategies and production maintenance runbooks
- Strong documentation and technical writing skills
- Experience working within formal change management processes
- Excellent troubleshooting and analytical abilities
- Ability to independently lead complex infrastructure projects
- Strong verbal and written communication skills
- Ability to perform structured root cause analysis
- Experience supporting mission-critical production environments
- Ability to work during scheduled after-hours maintenance windows
- Strong organizational and documentation skills
- Palo Alto Networks Certified Network Security Engineer (PCNSE)
- Palo Alto Certified Network Security Administrator (PCNSA)
- Experience supporting enterprise environments with 24x7 operational requirements
- Experience with enterprise VPN environments
- Experience supporting Federal Government or highly regulated environments
- ITIL Foundation certification
- Security+ CE (preferred)
Company Overview