ThirdParty

Differences between revisions 36 and 37
Revision 36 as of 2016-06-03 14:07:26
Size: 15555
Editor: localhost
Comment:
Revision 37 as of 2016-06-03 16:39:00
Size: 16174
Editor: breno-leitao
Comment:
Deletions are marked like this. Additions are marked like this.
Line 257: Line 257:
== IBM C/C++ Compiler ==

A key strength of XL C/C++ for Linux is its ability to generate highly optimized code for execution on IBM Power Systems. The performance gain from years of IBM compiler optimization experience on XL C/C++ for Linux can be seen in the release-to-release compiler improvements from the development of the IBM POWER4 processors through to the IBM POWER5, IBM POWER5+, IBM POWER6, IBM POWER6+, IBM POWER7, IBM POWER7+, and IBM POWER8 processors

[[http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=897&letternum=ENUS214-411|IBM XL C/C++ for Linux]]
== IBM C/C++ and Fortran Compiler ==

A key strength of the XL C/C++ for Linux (on little endian distributions) and XL Fortran for Linux (on little endian distributions) compiler products is their ability to generate highly optimized code for execution on OpenPower and IBM Power Systems™. The performance gain from decades of IBM compiler optimization experience on the XL compiler family which include XL C/C++ for Linux and XL Fortran for Linux is apparent in the release-to-release compiler improvements up to and including the current IBM POWER8® processor. XL C/C++ for Linux (on little endian distributions) now offers an alternative no-charge community edition for developer and partner use. IBM makes a Ubuntu repository available for XL compilers at: [[http://public.dhe.ibm.com/software/server/POWER/Linux/xl-compiler/eval/ppc64le/]] - visit here to get the installation commands.

For more information about the latest XL compiler release, see the Announcement letter at [[http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=an&subtype=ca&appname=gpateam&supplier=897&letternum=ENUS216-157]]

For more information about the XL compiler on Power, please visit its community at [[http://ibm.biz/xl-power-compilers.]]

Open Source tools

IRODS

The Integrated Rule-Oriented Data System (iRODS) is open source data management software used by research organizations and government agencies worldwide. iRODS is released as a production-level distribution aimed at deployment in mission critical environments. It virtualizes data storage resources, so users can take control of their data, regardless of where and on what device the data is stored. As data volumes grow and data services become more complex, iRODS is increasingly important in data management. The development infrastructure supports exhaustive testing on supported platforms; plug-in support for microservices, storage resources, drivers, and databases; and extensive documentation, training and support services.

Download page

MariaDB

There are some POWER8 special optimized binaries for MariaDB available at MariaDB site. For more information, check their instructions website

LuaJIT

We have a port of LuaJIT for ppc64el that was not already accepted by the luaJIT maintainer, but you can download it (source and binaries) from GitHub LuaJIT PPC64 page or the PPA page

When comparing SciMARK benchmark, Luajit performs almost 150% better than lua version 5.2.

Torch7

Torch7 can be installed by following the instructions here.

Mongodb, Nodejs, V8, Hadoop, Cassandra and general software

There is a page at IBM that describes all these major open source software and if they are supported on IBM POWER processor running Linux.

Saltstack

SaltStack is a new approach to infrastructure management. Easy enough to get running in minutes, scalable enough to manage tens of thousands of servers, and fast enough to communicate with them in seconds.

Salt delivers a dynamic communication bus for infrastructures that can be used for orchestration, remote execution, configuration management and much more.

Installing Saltstack

SaltStack is available in Ubuntu, but could be installed from github also.

From Ubuntu

# apt-get install salt-minion # On the minion nodes # apt-get intsall salt-master # On the master node

From upstream

In order to do it, do the following on all machines you want to have controlled by saltstack as in the master node:

   # sudo apt-get install python-m2crypto  python-yaml python-m2crypto  python-zmq python-msgpack python-jinja2 python-dev python-pip git
   # pip install pycrypto
   # git clone https://github.com/saltstack/salt.git
   # cd salt
   # sudo python setup.py  install

Start daemons

Once you have saltstack installed on all machines, you need to start the daemons.

For the minions, you need to specify which is the master node. The easier way to do it is configuring the hostname 'salt' point to the node IP, as for example:

# echo '10.1.1.4        salt' >> /etc/hosts

Then, you are able to start running the daemons. On the master node, you should run # salt-master or use the salt.master.service systemctl service. For minions, you should call # salt-minion, or, again, the proper salt.minion.service service.

Examples:

On master

Start the server with the following command:

salt-master

Or, preferably using the daemon services, as the following line for systemd:

systemctl enable salt-master.service

On Minions

For every minion you have, start the salt daemon with:

salt-minion

Or, preferably using the daemon services, as the following line for systemd:

systemctl enable salt-minion.service

Accept the minions keys in the master

When the minion connects to the master node, it provides a key that needs to be accepted by the master. In order to see the keys that are pending approval, go to the master node and use the command # salt-key -L, and to approve then, use the # salt-key -A.

After approval, you are going to see something like this, where ubuntu1404 and ubuntu140402 are my current minions.

 sudo salt-key -L
Accepted Keys:
  ubuntu1404
  ubuntu140402
Denied Keys:
Unaccepted Keys:
Rejected Keys:

This example shows that my both minions have the keys accepted.

After that, you can check to see if the connection is fine with your minion, using the following command:

# salt '*' test.ping
ubuntu1404:
    True
ubuntu140402:
    True

In this case, means that both minions are reachable and you go proceed and start using salt.

One of the example, is capturing the MAC address for the eth0 interface on all minions:

sudo salt '*' network.hw_addr eth0
ubuntu1404:
    4c:45:42:45:01:01
ubuntu140402:
    4c:45:42:45:01:09

Or, restarting all the minions at once:

sudo salt '*' system.reboot
ubuntu140402:
    
ubuntu1404:

IBM Tools

PowerVM tools

There are a set of IBM tools to install on Ubuntu when using Ubuntu over a PowerVM LPAR.

These tools could be found at IBM Fix Central Page.

Packages available:

  • src_3.2.0.6-15111_ppc64el.deb
  • rsct.core.utils_3.2.0.6-15111_ppc64el.deb
  • rsct.core_3.2.0.6-15111_ppc64el.deb
  • devices.chrp.base.servicerm_2.5.0.1-15111_ppc64el.deb
  • dynamicrm_2.0.1-3_ppc64el.deb

Some of these packages are also available from this PPA.

Advance Toolchain

The Advance Toolchain is a set of development tools, i.e. compiler, libraries, debugger and profiling tools, that help developers take leading edge advantage of IBM's latest POWER hardware features.

The following steps will guide you on the installation process of the Advance Toolchain. Detailed information is available in the Advance Toolchain documentation.

Firstly, import the Advance Toolchain GPG key by running:

wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/precise/6976a827.gpg.key
sudo apt-key add 6976a827.gpg.key

Note: In the following steps, replace atX.X by the desired version, e.g. at8.0.

Native installation (on POWER)

Add the following line to /etc/apt/sources.list:

deb ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty atX.X

Then, install the packages by running:

sudo aptitude install advance-toolchain-atX.X-runtime \
                      advance-toolchain-atX.X-devel \
                      advance-toolchain-atX.X-perf \
                      advance-toolchain-atX.X-mcore-libs

Cross compiler installation (on Intel)

The Advance Toolchain supports cross compilation from Intel processors and generating binaries for POWER processors.

Add the following line to /etc/apt/sources.list in your Intel-compatible computer:

deb [arch=i386] ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty atX.X 

Then, install the packages by running:

sudo aptitude install advance-toolchain-atX.X-cross-ppc64 advance-toolchain-atX.X-cross-ppc64le

For more information on how to use the Advance Toolchain cross compiler visit this page.

IBM SDK for PowerLinux

The IBM Software Development Kit for Linux on Power (SDK) is a free, Eclipse-based Integrated Development Environment (IDE). The SDK integrates C/C++ source development with the Advance Toolchain, Post-Link Optimization, and classic Linux performance analysis tools, including Oprofile, Perf and Valgrind. It is available as two different packages, providing you with three different development modes:

  • The IBM SDK for PowerLinux server development package allows you to use the IBM SDK for PowerLinux, which integrates the Eclipse integrated development environment (IDE), directly on the Power Systems server.

  • With the IBM SDK for PowerLinux client for remote development package, you can start the IBM SDK for PowerLinux x86_64 client, including the Eclipse IDE, on a workstation. You can then connect to a remote Power Systems server that has the IBM SDK for PowerLinux server agent installed to build, execute, debug, and analyze programs.

  • With the client for remote development package and cross-compiler feature, you can develop and build your application locally using the IBM Advance Toolchain for PowerLinux cross compiler.

To install IBM SDK for PowerLinux on Ubuntu, follow the instructions available at Download and Installation.

IBM Java

http://www.ibm.com/developerworks/java/jdk/linux/tested.html

Adding IBM Java Apt Repository

IBM Java is currently located into the Canonical partner archive and can be installed through the apt tool.

The partner repository is usually already into the file /etc/apt/sources.list, although disabled (commented). In order to enable it, one should uncomment or, in case it is not in the repositories list, add it to /etc/apt/sources.list.

   1 deb http://archive.canonical.com/ubuntu utopic partner
   2 deb-src http://archive.canonical.com/ubuntu utopic partner

Then retrieve the updated package lists:

   1 sudo apt-get update

Finally install it, via apt-get install. For the jdk:

   1 sudo apt-get install ibm-java71-jdk

Or jre:

   1 sudo apt-get install ibm-java71-jre

IBM C/C++ and Fortran Compiler

A key strength of the XL C/C++ for Linux (on little endian distributions) and XL Fortran for Linux (on little endian distributions) compiler products is their ability to generate highly optimized code for execution on OpenPower and IBM Power Systems™. The performance gain from decades of IBM compiler optimization experience on the XL compiler family which include XL C/C++ for Linux and XL Fortran for Linux is apparent in the release-to-release compiler improvements up to and including the current IBM POWER8® processor. XL C/C++ for Linux (on little endian distributions) now offers an alternative no-charge community edition for developer and partner use. IBM makes a Ubuntu repository available for XL compilers at: http://public.dhe.ibm.com/software/server/POWER/Linux/xl-compiler/eval/ppc64le/ - visit here to get the installation commands.

For more information about the latest XL compiler release, see the Announcement letter at http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=an&subtype=ca&appname=gpateam&supplier=897&letternum=ENUS216-157

For more information about the XL compiler on Power, please visit its community at http://ibm.biz/xl-power-compilers.

Adding IBM XL Apt Repository

IBM also makes a Ubuntu repository available for IBM XL at http://public.dhe.ibm.com/software/server/POWER/Linux/xl-compiler/eval/ppc64le/.

You can install the repository, running the following commands:

wget -q http://public.dhe.ibm.com/software/server/POWER/Linux/xl-compiler/eval/ppc64le/ubuntu/public.gpg -O- | sudo apt-key add -
echo "deb http://public.dhe.ibm.com/software/server/POWER/Linux/xl-compiler/eval/ppc64le/ubuntu/ trusty main" | sudo tee /etc/apt/sources.list.d/ibm-xl-compiler-eval.list
sudo apt-get update

IBM XL Fortran Compiler

A new XL Fortran compiler is delivered to support application development on the IBM POWER8 servers that run the little endian Linux distributions.

A key strength of the new XL Fortran for Linux is its ability to generate highly optimized code for execution on IBM Power Systems. The performance gain from years of IBM compiler optimization experience on XL Fortran for Linux can be seen in the release-to-release compiler improvements from the development of the IBM POWER4 processors through to the IBM POWER5, IBM POWER5+, IBM POWER6, IBM POWER6+, IBM POWER7, IBM POWER7+, and POWER8 processors.

IBM XL Fortran for Linux

IBM Engineering and Scientific Subroutine Library (ESSL)

ESSL is a collection of state-of-the-art mathematical subroutines specifically designed to improve the performance of engineering and scientific applications on the IBM POWER™ processor-based servers and blades. ESSL and Parallel ESSL are commonly used in the aerospace, automotive, electronics, petroleum, utilities and scientific research industries

Others

NVIDIA CUDA

CUDA 5.5 For IBM POWER8 is supported either in Ubuntu 14.04.

CUDA 7.5 is also available for POWER8 machines.

DPDPK

DPDK is a set of libraries and drivers for fast packet processing. It was designed to run on any processors, knowing Intel x86 has been the first CPU to be supported. Ports for other CPUs like IBM Power 8 or EZchip TILE-Gx are under progress. It runs mostly in Linux userland. A FreeBSD port is available for a subset of DPDK features.

http://www.dpdk.org/

MOFED

Mellanox OpenFabrics Enterprise Distribution for Linux (MLNX_OFED) Overview

Clustering using commodity servers and storage systems is seeing widespread deployments in large and growing markets such as high performance computing, data warehousing, online transaction processing, financial services and large scale web 2.0 deployments. To enable distributed computing transparently and with maximum efficiency, applications in these markets require the highest I/O bandwidth and lowest possible latency. These requirements are compounded with the need to support a large interoperable ecosystem of networking, virtualization, storage, and other applications and interfaces. The OFED from OpenFabrics Alliance (www.openfabrics.org) has been hardened through collaborative development and testing by major high performance I/O vendors. Mellanox OFED (MLNX_OFED) is a Mellanox tested and packaged version of OFED and supports two interconnect types using the same RDMA (remote DMA) and kernel bypass APIs called OFED verbs – InfiniBand and Ethernet. 10/20/40Gb/s InfiniBand and RoCE (based on the RDMA over Converged Ethernet standard) over 10/40GbE are supported with OFED by Mellanox to enable OEMs and System Integrators to meet the needs end users in the said markets.

http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers

ppc64el/ThirdParty (last edited 2018-05-08 19:09:28 by rogerioalves25)