Thursday, February 19, 2015

More random thoughts about bartending

After a season spent doing 1-2 shifts a week at both of the bars as well as the coffee house/wine bar, a few more thoughts on this bar scene of which I understand nothing:

-If you ask me to recommend a wine, I'm going to suggest whatever we have with a screw-cap. Corks are stupid; your fancy tradition is my pain in the ass. 

-After having overheard far too many drunk conversations, I'm never trusting any guy who wants to travel solo to Southeast Asia. 

-ARG STUPID FSKING BRAND NEW DOLLAR BILLS I HATE YOU SO MUCH.  You're the reason my drawer was off at the end of the night, you're always getting stuck together and are impossible to quickly count. 

-No, you cannot send a drink to that girl at the end of the bar. She is half your age, a third of your weight, and I'm damn sure has no idea who you are. Hit on someone your own size. 

-Seriously, you see how most of the girls are clustered together in groups? That's to try and avoid guys like you. I'm not doing your dirty work of trying to make introductions (although I will laugh at your resulting rejection). 

Saturday, February 14, 2015

Pistin Bully

The warmest times of the year down here is late December through January, when plentiful sunshine can push the daily temperatures into the high 20s.  The winds during these months are usually calm and when this is combined with solar action on the dark colored soil, most of the snow around station has melted out and you can take a normal wheeled vehicle anywhere.

Almost anywhere.

If you've got to go up to the radio repeaters on top of Crater Hill overlooking station, the only option is crawling up a 40% grade of loose rocks and gravel.  And for trips like that, the vehicle we turn to is a steel-tracked Pistin Bully.

Little more than two boxes on a hydraulically driven tread system, Pistin Bullys are the final work on station for ground transportation that will get you absolutely anywhere you need to go (as long as you don't need to go more than 5mph).  If you've gotta be someplace that can't be reached by these guys, nothing else short of a helicopter will get you there..

The driver's compartment in the front is separate from the passenger/cargo compartment in the back.  It's roomy enough, but not what you'd call luxuriously equipped.

In spite of being separated by not more than two panes of sliding glass, there's an intercom phone you can use to call the driver's compartment 18 inches away.  We mostly used it to annoy them.

"Yeah, can I get a large pepperoni, an order of breadsticks, and . . . you deliver, right"
A journey up to the Windmills/Beach Ball at T-Site takes 5 or 10 minutes in a pickup truck, but crawling along in a PB turns that into nearly an hour-long trip.  And it's not an easy hour; it's the roughest ride you can possibly imagine. There's no suspension on these things, and not even a rubber tire of some kind to soften out the jarring from every little pebble along the way.  It's steel treads, along steel wheels, to a steel frame, to your spine.

Once you arrive at T-Site (which can be reached by any vehicle), you continue past the wind generators and start up the hill.  You can just make out the trail, by the line of accumulated snow heading upwards to the large patch of snow at the top.

Once you start the hill climb, the rear compartment passengers have to switch over to the forward bench seat and brace yourself against the other seat, and use the bars on the ceiling as hand-holds to hold yourself in place.  The PBs are so short and the slope so steep that you have to keep all the wight forward in the vehicle, to prevent it from tipping over backwards.

While trying not to get shaken to death, either.  It feels like an army of very angry people is pounding on the bottom of the vehicle with sledgehammers.

Looking back the way we'd came.

Once you reach the top of two intense hill-climb sections, there's a few hundred yards of following the trail along the ridge line before we get to our destination.

And slowly enough that I can pop out to play tourist for a bit

We're trying to get the equipment huts that hold much of our radio transmitting equipment.  It's not terribly dangerous, although the big metal plate that protects a fiber optic line can be harrowing to cross and usually requires a spotter.

We reach Hut #65 a few minutes later, and the Comms techs set about working on their equipment. 

The road ends here, but I've got a little farther to go.  From here it's on foot, scrambling up the ridge to the "Dog House", the end of this communications line.

A view from the ridge heading to the Dog House, with the PB to the left and McMurdo in the background

This ridge is the highest point around McMurdo station that is accessible by the station's hardwired network, and as such serves as the mounting point for many of our long-distance wireless communications.  The actual radios and transmitters are maintained by the Comms department, but my job is still to make sure the blinky light boxes get the data to the radios in the first place.

We don't bother with much high-tech security up here.  I think it's remote enough that we don't have to worry about hooligans.

Inside the Dog House is the usual mess of Telco, Comms and Network equipment.  The larger hut down the ridge line is kept at operational temps just by the heat given off by the electronics inside, but there's not enough of that up here.  What little heat our equipment puts off has to be conserved to try and keep it within it's operational range; a scrap of egg crate foam and some wire seems to do the trick.

It doesn't have to stay "Warm", it just has to stay above -40f.

I wonder if I could get a pizza sent up here...

As much as I enjoy getting off station and doing things that don't involve a desk, I think I could live without too many more PB rides.  The adventurous factor wears of pretty darn quickly, but the bruised shoulders, back and butt remind you of it for at least a few days after.

Saturday, January 31, 2015

Actually, it mostly is a desk job

In all honest, it really is a 95% desk job.  The vast majority of my days are spent staring at three monitors (and usually at least one laptop), often trying to make sense of multiple different bits of information coming in from different sources.

Network Administration isn't as much of a Break/Fix job as being a PC Technician or to a lesser extent, System Administrators.  PC Techs fix the computers, System Administrators fix the servers.  And I fix all the stuff that connects them together.  But if I'm lucky, the stuff I manage doesn't actually break that often.  Networking is more of a build-it-and-leave-it task, and anything that's completely broken is 99% exceedingly simple (someone unplugged a cable) or that 1% that is ridiculously convoluted (when a strange routing error was sending traffic from a specific server over the VLAN that should only be for our wireless access point management).  Most of my day is spent working on projects of various sorts, upgrading things, making tweaks and adjustments to better suit different situations, and poking at long-term projects.

So as for what that IT guy in your office who's always sitting on his butt and staring absentmindedly at his screens is actually doing, here's what's on my plate right now.

  • Statically assign some IP addresses to a bunch of dataloggers out in the field, then open specific holes in our firewall to allow science groups to access their equipment here on station from their universities back in the states.
  • Upgrade the operating systems on a handful of switches and routers that are on our separate "Monitoring and Control" network, which is used to manage our satellite connection.
  • Update the ACLs (Access Control Lists) and SNMP (Simple Network Monitoring Protocol) traps for those devices as well, to bring them in line with the rest of our network and allow us to monitor them better.
  • Write some documentation
  • Figure out how to move the MAC address filtering for some of our wireless networks off our WLC (Wireless LAN Controller) and onto our ACS (Access Control Server). This is particularly complicated as the devices that we're MAC filtering aren't on the domain, nor do the users have domain accounts.  I've got a few theories about how to accomplish this, it's a matter of testing them and figuring out which is the best for long-term manageability.
  • Rebuild most of our automated Cacti graphs that we use for monitoring bandwidth utilization at different points around our network, after a glitch last winter broke about 70% of them.
  • Write more documentation
  • Go over to our weather and aircraft control facility and wireshark the connections on the two computers that are supposed to be able to use our backup WAN link if our main connection goes down, and try and figure out the routing problem that's resulting in them sometimes using the backup connection for specific web sites even when our main connection is still up (Even though we've got the costs set high in EIGRP, for some sites it still seems to use the backup when trying to access our severs in Christchurch)
  • Replace a 24 port switch in another building with a 48 port, to allow for the eventual expansion of the office space and installation of a wireless network.
  • Figure out why our WLC isn't doing automated backups (probably an authentication issue between it an our FTP server)
  • Write even more documentation
  • Install some new WAPs (Wireless Access Points) into our bars and coffee house, for the new iPad-based Point-of-sale systems.
  • Trunk the dedicated Financial VLAN to the WLC, create a new virtual interface inside the WLC and set those Point-Of-Sale access points to send all data through that VLAN.  Also will need to configure a DHCP scope for it.
  • Configure that SSID for the POS system to be shared among all the APs, MAC filtered with a Pre-Shared Key for now.  But eventually I'd like to move authentication for it onto the ACS, because of it's vastly better security and manageability.
  • There's probably some things that still need to be documented, so I should write that.
  • Install a small switch and temporary wireless network into "Hut 10", the small house-like structure that some DVs (Distinguished Visitors, aka VIPs) will be staying in for a few days next week.
  • Figure out how to trace MAC addresses over wireless bridges, because my usual method of the show mac address-table address command doesn't work when the target is at the other end of a point-to-multipoint wireless network. (AH HA VICTORY I actually figured this one out mid-draft of this post.  Log into the wireless root bridge, use show bridge verbose | inc with the target mac address, which will give you the virtual interface that points in that direction.  Then show cdp neighbor vi# will identify the device on the other end of that virtual interface)
  • Oh, I figured out something new.  I should probably document that.

It's a lot of desk work.  An awful, awful lot of desk work, and that's one of the hardest thing so far about this job to get used to.  Most of my prior work experience in IT has been as a PC Tech of some kind, which has you frequently out in the field or at least moving stuff between different buildings. Even if you're not doing hard labor, you're still moving around.  But now I've had many days where the only time I get out of my chair is to go to lunch, or maybe go to a meeting where I'll try not to fall asleep.  For every hike up to the Beach Ball, or drive out to the Runway or Balloon facility, there's a solid week of sitting at my desk and pushing buttons.

Although now that it's warmed up a bit more, on the occasions I do heve to hitch a ride somewhere I can just jump in the bed of a pickup truck.

The thing that has surprised me most about this job, though, is how mentally exhausting it is.  Doing building maintenance at Palmer was a hard job, and there was some thinking involved, but it wasn't nearly as intensely cerebral as being a Network Admin.  As a guy-with-a-wrench I would get off work at the end of the day and my head would be buzzing with ideas and energy, sighting new photography or time-lapse opportunities or building some kind of craft project or writing up a new blog post.  But now I get to the end of the day and my brain just wants a rest; I've expended almost all of my creative energy at work and when it's done, I just want to consume information rather than create it.  I want to read, or play some games on my phone, or go bartend, just hang out in the Galley with my Stewie friends.  I stare at a screen for 9 hours a day now and when it's done, I find it difficult to conjure the energy and motivation to sit down and spend the few hours that a really good blog post actually requires (Hence, the once-a-month updates).

Let me be very clear: I love this job.  It's everything I wanted it to be, and so much more. It's fascinating to a degree that makes me a little bit embarrassed to admit; even I find it a little weird at just how excited I can get over a successful RADIUS authentication to our ACS, even if seeing that success is the culmination of four days worth of effort.

When I was a kid my friends and I were borderline obsessed with a computer game called Myst, which was an open-world exploration/puzzle solving game that was (and is still) famous for it's mind-bending difficulty.  It was a game where you might be exploring a building, and in this building you might find a machine with many levers, buttons and knobs, all of which did something but you had no idea what.  And with copious note-taking, lots of testing and a bit of luck (and the occasional "Ah-ha!" moment), you'd eventually figure out what each button and lever on the machine did, how to operate the machine, and how to get the machine to do what you wanted.  In the game, it was often something like "Manufacture a key to allow you to open this door".

And I realized a few weeks ago that my job is exactly like playing Myst.  I'm pointed at systems that while they do have a logic to them, it's often a unique logic and sorting out how it works is a long process of "Click on everything you can find, and see if you can start making connections between your inputs and the device's output".  And every so often you get those "Ah-ha!" moments and it's even more rewarding when it's something useful and practical in real life, rather than just a game.

I could see how this could drive some people insane; we've got extremely high personnel turnover year-to-year and documentation/knowledge transfer is often scant.  Quite a number of systems here were set up years ago by someone who may have been an expert at them, but they've long since left the program and no one has really touched them since.  People who are process-oriented, who thrive on order and predictability and who like knowing every step of a task before they start it would start bleeding from their ears in places like this.

But it's what I thrive on, and it's exactly how I learn.  I don't have much/any formal education; I'm not sure if I ever graduated from high school and I never went to college.  Outside of a couple certification courses that I took many years ago, everything I know I've taught myself, by stubbornly poking at something and taking it apart until I can find out how it works.

I love it.  I absolutely love it, and I can't wait to come back and continue the process next season.

(I promise I'll do a post soon that has actual pictures, rather than a random brain dump about my job that I'm a little bit too passionate about)