Linux Curl with Proxy

What is Curl?  curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.

So in the cloud you may find yourself on a server that needs to communicate to an external system but it isn’t allowed to go there directly.  In which case you will most likely have a proxy server to communicate through.  In PeopleSoft you can define a web gateway proxy server, however, to test to make sure that the server is working correctly with the proxy you will want to do a curl test.

Linux Command Prompt –> curl http://whatsmyip.com

This should return the external IP address of the system you are coming out of.  If this returns nothing you are likely blocked from going to that address.  So to push the request through the proxy server, you need to set the proxy in an environment variable:

Linux Command Prompt –> export http_proxy=http://myproxy.atmyserver.com:80

Linux Command Prompt –> curl http://whatsmyip.com

Now as long as the proxy is allowed to communicate to that address you should see the IP address of the system you are communicating from.

Oracle Sequences

Sometimes in the middle of the night you will be working on a conversion effort and you will need to populate a field with a sequence of values.  This is easily done using Excel but if you are dynamically loading data you may want to have a more flexible feature.  Oracle sequences have that ability.

Create A Sequence:

CREATE SEQUENCE adjustment_fix START WITH 1001 INCREMENT BY 1 NOCACHE NOCYCLE;

So this sequence will start are 1001 and each additional next value will by greater by 1 — so 1002, 1003, 1004….etc.  I had an oddball situation where I needed an 18 digit number that increased by 10000000. So in my case it was:

CREATE SEQUENCE adjustment_fix START WITH 300000000010000000 INCREMENT BY 10000000 NOCACHE NOCYCLE;

Now that you have the sequence, you need to populate the rows with the sequence.  So I created a temporary table and loaded all the values in the table.  Then I had another field for the sequence number.  I tried to insert the sequence while doing the insert into the staging table of the values I needed staged for the adjustment but it whined and complained.  So I did the insert and then did an update of the rows:

SQL:    UPDATE STAGING_TABLE_4_ADJ SET ADJ_SEQ = adjustment_fix.nextval;

BAM!  Now I have 2200 rows of adjustment data with a unique sequence number on each row so that I can load the staging data tables.

Panda + Community – August 2016

August ended up being a little bit quieter, which was actually kind of nice for a change of pace.

Team Panda was out at the Big Brothers Big Sisters of St Thomas Elgin Annual Golf Tournament.  We were not the most successful golfers on the course but we had a lot of fun and the weather was fantastic – so we ended up enjoying the walk around the course! 🙂

The Smart Panda (OurOnline.Company) is the title sponsor this year for our local Big Brothers Big Sisters, so it was great to get out and support them as they raised a significant amount of money to put towards their in school mentoring programs and also to support their Big/Little matches. Currently over a hundred kids are getting mentored, and there is no way to ever measure the impact of being involved in a young person’s life as they walk through the challenges of simply being a kid.  At The Smart Panda we work with the theory that you have to have a Trusted Partner (or Mentor) to get you through the challenges.

In the pictures below you will see a helicopter, which was also at the Golf Event and they dropped 2000 golf balls for Big Brothers Big Sisters “Big Ball” Drop.  The ball closest to the hole (or in the hole) won $2000.  The local helicopter training academy does the helicopter drop which is actually pretty cool to watch!

The Rotary Club is gearing up for a busy fall and winter season.  Once the weather starts to get cold we will be back helping Inn-Out-Of-The-Cold serving up great food and helping some very deserving people. The Rotary Club and The Smart Panda will be out working with Christmas Care again this year.  Wade is on the board of Christmas Care this year to help ensure that everybody at Christmas can have a great Christmas.

Just in case you still haven’t donated to the MS Ride – its NOT too late: Didn’t get a chance to Support the Ride?  IT’S NOT TOO LATE:  Click here to Support The Panda!

Until Next Month……

In one of my clients Oracle Managed Cloud Services environments which is extremely locked down, they were experiencing terrible performance issues trying to go to WB Mason’s external punchout site.  It was taking upwards of 5 minutes to return the catalog pages.

Something was CLEARLY wrong. In this case it was DTD – and what is DTD?  Well, document type definition of course – which is a set of markup declarations that define a document type for an SGML-family markup language (SGML, XML, HTML). A Document Type Definition (DTD) defines the legal building blocks of an XML document. It defines the document structure with a list of legal elements and attributes. A DTD can be declared inline inside an XML document, or as an external reference. XML uses a subset of SGML DTD. As of 2009, newer XML namespace-aware schema languages (such as W3C XML Schema and ISO RELAX NG) have largely superseded DTDs. A namespace-aware version of DTDs is being developed as Part 9 of ISO DSDL.[2] DTDs persist in applications that need special publishing characters, such as the XML and HTML Character Entity References, which derive from larger sets defined as part of the ISO SGML standard effort. (Yes, I cut and pasted that from wikipedia DTD page )

So basically the cXML of the external punchout is trying to validate against an external reference which the application server is not allowed to go to because of firewall restrictions. In this case because the only items we had validating were these valid external punchout sites, it seemed logical to turn off DTD validation! Which you can do since 8.49 PeopleTools. The version my client is on is 8.54, so it turns out that it is really easy in 8.54 to turn off DTD validation.

In the Web Integration Broker Gateway configuration (integrationgateway.properties) file, there is a setting for DTD Validation, simply change this to “false” from “true” and reboot the web server and what was once extremely slow is now returning in fractions of a second.

There are other ways to work around this issue, but this is a relatively quick item you can test to validate if DTD is causing your performance issues.

Panda + Community – July 2016

We cooked and Peddled our way through the month of July and had an absolute blast doing so.

The Panda got to celebrate birthday #42 by cycling 154 km (~100 miles) from Grandbend Ontario to London Ontario and back. We are part of team Butt Ugly and we have raised almost $30,000 this year to help end MS.  Currently the entire event has raised 1.16 million with the target being 1.3 million which we will definitely reach once all the dollars are counted.  Thank You, Thank You, Thank You to everybody that supported us out there.

The St Thomas Railway City Rotary Club was out BBQ’n up some awesome hot-dogs and sausages to raise funds for local community projects this month.  A lot of onions and ketchup were dished out making for some very happy customers.  All the funds raised will be going back into the community this year as we gear up to support projects like Inn-Out-Of-The-Cold which helps the homeless in our community survive the winter with food and shelter or the local conservation authority which we have pledged to purchase and plant 100 more mature trees to help the environment and make our wood lots more vibrant and healthy.  The Panda even got put in charge of running the meetings for the upcoming month as our fearless leader got off his bike at the MS ride and jumped on a plane to England to go cycle another couple hundred miles around England (yes – he is crazy).

August we are going to be busy gearing up to support Big Brothers Big Sisters with their annual golf tournament and big-ball drop.  The Smart Panda is a proud supporter of Big Brothers Big Sisters of St Thomas Elgin, because we feel everybody deserves to have an amazing mentor.

Didn’t get a chance to Support the Ride?  IT’S NOT TOO LATE:  Click here to Support The Panda!

Until Next Month……