How many SmartConnect Zones on Isilon for HDFS

There appears to be some confusion here; the older Isilon Hadoop docs recommended separate subnet pools for the NameNode (NN) and the DataNodes (DN), this changed recently and now the suggested recommendation is a single subnet pool for NameNode (NN) and DataNodes (DN), the pool should be dynamic to provide clean failover of HDFS services. The use of Isilon virtual rack requires more pools but more on that later.

In the sample configuration below; we have two pools, a static pool for stateful workflows [workflow1, this is non hdfs, let’s say it’s your smb workflow pool] and a second dynamic HDFS pool [HDFS] but you could get away with a single dynamic pool if needed.

see the latest Isilon Hadoop Best Practice doc for more:

https://www.emc.com/collateral/white-paper/h12877-wp-emc-isilon-hadoop-best-practices.pdf

 

# isi networks list pools –verbose

subnet0:pool0 – workflow1

In Subnet: subnet0

Allocation: Static

Ranges: 1

10.0.0.21-10.0.0.24

Pool Membership: 4

1:10gige-2 (up)

2:10gige-2 (up)

3:10gige-2 (up)

4:10gige-2 (up)

Aggregation Mode: Link Aggregation Control Protocol (LACP)

Access Zone: System (1)

SmartConnect:

Suspended Nodes  : None

Auto Unsuspend … 0

Zone : isl-wkfl1.isilontest.foo.com

Time to Live     : 0

Service Subnet   : subnet0

Connection Policy: Round Robin

Failover Policy  : Round Robin

Rebalance Policy : Automatic Failback

 

subnet0:pool1 – HDFS

In Subnet: subnet0

Allocation: Dynamic

Ranges: 1

10.0.0.25-10.0.0.50

Pool Membership: 4

1:10gige-2 (up)

2:10gige-2 (up)

3:10gige-2 (up)

4:10gige-2 (up)

Aggregation Mode: Link Aggregation Control Protocol (LACP)

Access Zone: hdfs (2)

SmartConnect:

Suspended Nodes  : None

Auto Unsuspend … 0

Zone : isl-hdfs.isilontest.foo.com

Time to Live     : 0

Service Subnet   : subnet0

Connection Policy: Round Robin

Failover Policy  : Round Robin

Rebalance Policy : Automatic Failback

 

This isl-hdfs.isilontest.foo.com is SmartConnect name used when connecting the compute cluster to the Isilon for HDFS access.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s