01 July 2014

APEX_ESCAPE, a new (and better) way of HTF.ESCAPE_SC

Last week, at the yearly ODTUG Kscope Conference, I did my presentation "Getting Started with APEX Plugin Development". After the session Patrick Wolf, Principal Member of Technical Staff for APEX, pointed out an improvement that could be made.
In the presentation I point out the need to escape the input that you get from a user of the plugin in order to protect the plugin from unwanted use, like SQL Injection, Cross Site Scripting and the like.
In the example plugin that is created in the presentation, I use HTP.ESCAPE_SC to escape the special characters (hence the name _SC). There is a newer and better method to escape the special characters.
By default the extended level of escaping is enabled, but this can be overridden (for whatever reason).
To illustrate both the extended and the basic level of escaping, the examples below set the level explicitly.
SQL> begin
  2     apex_escape.set_html_escaping_mode (p_mode => 'E');
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> select sys.htf.escape_sc ('hello &"<>''/') htf
  2       , apex_escape.html ('hello &"<>''/') escape
  3    from dual
  4  /

HTF                            ESCAPE
------------------------------ ----------------------------------------
hello &amp;&quot;&lt;&gt;'/    hello &amp;&quot;&lt;&gt;&#x27;&#x2F;
With the extended level of escaping, the forward slash and the single quote are escaped as well.
When you set the escaping level to Basic (example below), you will get the same results as if you were using HTF.ESCAPE_SC.
SQL> begin
  2     apex_escape.set_html_escaping_mode (p_mode => 'B');
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> select sys.htf.escape_sc ('hello &"<>''/') htf
  2       , apex_escape.html ('hello &"<>''/') escape
  3    from dual
  4  ;

HTF                            ESCAPE
------------------------------ ----------------------------------------
hello &amp;&quot;&lt;&gt;'/    hello &amp;&quot;&lt;&gt;'/

Links

Documentation on APEX_ESCAPE

26 June 2014

ODTUG Kscope 2014: Wednesday

The morning started nice, ODTUG organized breakfast with country themes. There were tables for Australia, Canada and The Netherlands. If you are Dutch you might have expected to have hagelslag or cheese, this was not the case. It was a nice and hearty American style breakfast, eggs, bacon, potatoes.
The first session of the day that I went to was by Nathan Catlow on Oracle APEX Security, an interesting topic.
Nathan pointed out that by far the most common security has to do with Cross Site Scripting (XSS). This can lead to data protection issues, account compromise and attack of other applications.
Regarding injection attacks, Nathan pointed out that substitution variables (&P...) in comments are also prone for Injection attacks.
Very good advise to upgrade to at least APEX 4.2.1. There are vulnerabilities in the APEX framework which are fixed in this release.
The next session was another one on APEX Security by Tim Austwick, this time with a focus on SQL Injection.
Lots of practical information regarding SQL Injections. After listening to this, it makes you wonder how secure applications are. On the other hand it is good to known that I implement loads of their advise already. :)
"Pins Polygons and Perspectives: Visualizing Geographic Data in APEX" by Christoph Ruepprich was next.
One of the mapping apis that I never heard of was LeafletJs. Looks really nice, yet another thing to put on my to-do list.
After lunch I attended Jonathan Lewis' session on the Cost Based Optimizer for Developers. The session was very well attended and the content was superb.
According to Jonathan Oracle must obey your index hints, but only if you get it absolutely correct. If you tell the wrong path, you left out information (hint missing) or if you tell Oracle to do something "illegal" than Oracle will not follow your hints.
John Scott did a presentation about NodeJs. You can expect to see a lot of demos when John does a presentation and this time was no exception. Besides the installation of NodeJs, he also installed node-oracle for the connection to the database.
Demos included Grunt, Mail-listener2, Officegen and pdfkit. In one word: awesome.
The last session of the day: Dimitri Gielis on his way of developing APEX applications. A very useful tip that Dimitri shared was to create a template application so you can have a nice starting point for the application. You define the basic building blocks (like global page, lists, administration page, include font-awesome) and export the application. Then go to the internal workspace and add this application as template application.

Tonight there is the traditional party, this time it will be at the Seattle EMP (experience music project). Just realized that there is an Oracle link there... (emp as the table in the Scott demo schema - just the geek in me I guess). ODTUG has a reputation to uphold regarding the parties, so I expect a lot from it.

25 June 2014

ODTUG Kscope 2014: Tuesday

Mark Drake, the product manager for XMLDB and the new JSON features in the database, started Tuesday with a session on flexible storage.
After a short history of the XMLDB ("more than just a LOB Store"), he went to the heart of the presentation: flexible storage.
In the upcoming release of the database there will also be JSON functionality built in. The JSON functionality won't have a separate datatype. By not introducing a separate JSON datatype the implementation of replication and high availability won't be as hard to implement as with the XMLType. On the other hand, it is harder for the API to figure out which datatype it should project to.
There are several ways to implement flexibel storage, like:
  1. flex-fields
  2. document persistence
  3. name value pairs
With flex-fields you have no way of knowing the content of the data by looking at the datamodel. By using document persistence you would implement this with content stored as XML or JSON.
Some problems can be solved when you represent name value pairs as XML. This solution was described in detail, including indexing strategies.
Right before lunch I did my session "Getting started with APEX Plugin Development". There were about 45 people in the room and I think the session went allright. After the session I had lunch with Patrick Wolf and we were going over some of the different aspects of the plugins and he gave me a few good pointers. More stuff to play with and figure out how they work.
OTN (Oracle Technology Network) sponsored a lunch and learn session. In a packed room there were some very good questions and answers.
Dietmar Aust covered some "small" features of APEX 5. This time the "Page Designer" was not the main focus of the presentation, which was a big part in other sessions on APEX 5. Instead Dietmar discussed the change in the export functionality, the different ways modal pages can be created and session joining, just to name a few.
Not new in APEX 5, but might be useful: APEX_MAIL.GET_IMAGES_URL and APEX_MAIL.GET_INSTANCE_URL. Go find more about this in the documentation.
Instead of using the v-function, use the context functions like SYS_CONTEXT ('APEX$SESSION', 'app_user') to get information about the APEX context.
There is also a extended method to escape substitution variables, e.g. &P16_EMP_NO!JS. He promised that he will blog about it.
There is a new APEX_ZIP package which is based on the AS_ZIP package, created by my former colleague Anton Scheffer. There will also be an APEX_JSON package with lots of functionality.
Finally there is an alias APP_PAGE_ALIAS, long overdue.
Next up Peter Raganitsch on things you can learn from the Packaged Applications which are created by Oracle. A lot of very nice plugins are hidden inside these packaged applications.
Peter also demonstrated a method of installing packaged applications using the command line as opposed to using the wizards. Unfortunately using the command line doesn't install the supporting objects, so the application doesn't work. This still needs to be done using the regular method in the builder. So how did he got it to work? He imported the APEX Builder (which is just an APEX application after all) and found the wizard which install the supporting objects.... just don't do it, there are some undocumented API's being used.
As there are a lot of plugins (around 50) in the packaged applications, Peter also demonstrated a way to export these by utilising the java command line tools (APEXExport and APEXExportSplitter) to export and split the applications.
The final session of the day that I attended was by David Mann: "Time for some New Graphs: Incorporating Time and Animation Elements in your Web App Visualizations".
David showed different types of visualisation, some very funky ones, some very useful. He showed some demos using D3. Personally I had never heard of D3, so definitely something to play with in the near future (at least I hope near future).
At night the Oracle ACE program organized an ACE Dinner. It was at the Pike Brewery, needless to say there was some beer involved with a very nice meal.

ODTUG Kscope 2014: Monday

Monday morning started with the opening keynote. At the same time the Dutch soccer team played Chile (which the Dutch won with 2-0). A small Dutch delegation went to watch the game, all dressed in orange, I was one of them.
During the keynote the location of Kscope 2015 was announced: Hollywood, Florida.
Lunch was special, there were lunch and learn. There were "reserved" tables were you could sit on and talk tech with ACE Directors. We had some nice discussions at the table that I sat on, including the soccer game.
After lunch I went to see a session by Kris Rice on the Oracle Rest Data Services (formely known as the APEX Listener) Best Practices and Features.
The first (and the last) thing he pointed out that should must change the following settings:
  1. Configure database connection pool
  2. set max size
  3. set initial size
  4. set timeouts
Kris also spend some time on ICAP which stands for Internet Content Adaptation Protocol. This will scan all file uploads for viruses before it reaches the database. They needed that for the Oracle Cloud services.
In the APEX Listener you could use PL/SQL for URL validation, nowadays you can also use Javascript for these validations. Even though the demo didn't go as smoothly as Kris would have wanted to, the point was clear. There is no database hit, it is handled in the ORDS.
The Rest filtering option looks really interesting. With these rules you can add filtering options before you do down to the database. Definitely something I need to look into more closely, it sounds very interesting.
There is also a way to log all PL/SQL call and the bind variables, there is a sample_capture.sql (somewhere). It needs to have a certain signature which is specified in that sample script.
For version 3.0 there are a lot more features coming, like "Simplified Installation", "Client REST Filtering", "Bulk CSV Loading over REST", "Database 12.1.0.2 JSON Collections", a "New Plugin framework", and more...
The next session I went to was another one by Kris Rice, this time it was about "Creating RESTful APIs with Oracle Application Express Listener".
There are over 40 internal group at Oracle defining the REST standard. This means that if you know and understand how to interact with REST and ORDS, you also know how to interact with REST and Fusion.
After an introduction about what REST Data Services is all about, Kris continued with lots of demos including "SQL Injection as a Service".
In ORDS version 3.0 you can "REST enable" the table with a single click from SQLDev - very cool.
The final (regular) session of the day was by John Scott on "Testing APEX: removing the Boring from Testing".
He started of by comparing testing to Marmite, you either love it or you hate it.
The most fundamental of tests is Unit Test, followed by Integration Tests. Above that are Functional Tests and finally the Acceptance Tests.
John covered NodeJs, PhantomJs, CasperJs, SlimerJs and TrifleJs to help with testing. He did the demos with CasperJs.
He made a good point, and I am really interested in trying out CasperJs to do some testing.

24 June 2014

ODTUG KScope14 Sunday APEX Symposium

The ODTUG Kscope conference always starts with a full day symposium. There are several specialized tracks going on, and I attended the APEX track (mainly).
The room was packed, I would estimate around a 175 delegates.
Joel Kallman started the day with an overview of the history of APEX, including a video of Steve Balmer being very enthousiastic about APEX. He also told a bit about the background and how some features evolved throughout the years, like Themes and lessons learned with the packaged applications.
Joel showed a couple of videos demonstating the Page Designer vs. the current wizard driven style of development. Loved the new video with the song "Everything is Awesome" to show of the productivity-boost by using the Page Designer in APEX 5.0.
Deployment also gets easier with APEX 5. Now you can associate the database objects with your application. The DDL is then generated, so you don't have to go to external tools to get the DDL.
Joel also shared a story on the usage of APEX as mission critical application. There was a large customer who experienced a production outage on a mission critical application after an upgrade. Turned out to be a database bug (outline),but it shows that APEX is being used by large organizations and in mission critical applications.
One of the last things Joel shared was that APEX will get more marketing in the future to get more companies using APEX.
Next up Patrick Wolf about the Page Designer. The demo's that Patrick did really showed how the Page Designer improves developer productivity.

After lunch Shakeeb Rahman gave some insight in the process that was taken by the redesign of APEX 5. He also spend a good amount on the Universal Theme, especially on Theme Styles and Template Options.
Without touching any HTML create a new look for your applications by using Template Options, very flexibel. Just by changing some options, you can change your applications dramatically
Font Awesome is included standard, 400 icons to choose from which can be easily incorporated. For now there is no quick-pick available, but a future release might include that as well.
All the templates and regions will be shown in a sample application,like the eight permutations of a list template. Unfortunately this sample application is not yet available.
With Theme Styles it simply a flip of a switch to change the color scheme of your application. Currently Universal Theme 42 comes with Blue (Default) and Red.
One more thing: Universal Theme Roller, and again the song "Everything is Awesome" is played.
The Universal Theme Roller works (more or less) like the jQuery Themeroller, but now on your own application. Of course Shakeeb picked Orange for the demo. The changes made in the Universal Theme Roller popup where shown live in the application (using Less) and the setting stored in the browser.
The CSS can be saved to a file and included in your application. Hopefully there will be a possibility to save the CSS directly in your application.
The Universal Theme Roller is not available in the EA2, but they are working hard on it to include this.
Jason Straub did a presentation on APEX and the Multitenant Architecture of the 12c Oracle Database. This session was at the same time that the USA played Portugal in the world cup, still the room was quite busy.
The session started with an overview of the Multitenant Architecture, followed by all the different options that you have were to install APEX. There are a lot of different scenarios with copying and moving PDBs between different Oracle 12c databases. Thankfully Oracle provides a lot of scripts which help you with all of these scenario's.
And that wraps up the first day of ODTUG KScope14.

11 June 2014

Finland: OUGF Harmony 2014

Last week was the annual conference for the Oracle User Group Finland. This time the conference was at the Finnish Nature Center called Haltia. Needless to say that the location was beautiful.
Even though the conference started on Thursday, most speakers flew in on Wednesday. Heli, the president of the OUGF and dear friend, acted as a tourguide and showed Helsinki. The tour ended with a dinner at a Viking restaurant where the food was very good. We got to try some tar ice-cream,.. Tar, yes tar - as in the stuff that you find in cigarettes. It tasted like it should be hot, black and oily, instead it was white and cold. Not my thing.
The next morning busses were taking all the delegates from Helsinki to Haltia, about a 30 minute drive.
After breakfast and the opening keynotes, there was a choice of 6 tracks.
The one that stood out for me was done by Brendan Tierney on Sentiment Analysis. Certainly worth it to investigate a little more.
In the evening there was a BBQ with plenty of salmon, reindeer burgers and strawberry cake. There was also a band playing made up of Finnish Oracle employees.
Most delegates were bussed back to Helsinki, only a few "braves" stayed to go camping in the local forrest. The strange thing is that it didn't really get dark, just dusky. The second picture of the camping was taken around midnight.
Even though they warned about the Finnish mosquitos ahead of time, obviously I didn't take enough precautions. I got bitten quite a few times. All part of the experience. The camping was great - thanks Heli, Ann and Olli for taking care of us.
The second day of the conference I had both my sessions. The first one (Use Cases for Virtual Columns) was right after Graham Woods keynote. He was running a bit late and I had the same room, so I started late.. Anyhoo, attendance was good and I thought the session went alright.
My other session (SQL Model Clause: a Gentle Introduction) was the last session of the day, right before Heli's closing keynote.
It is always amazing to listen to Heli, she speaks Finnish and English and switches all the time.
On Saturday, before we left for the airport, I did some sightseeing with Kent Graziano and Roel Hartman. I am glad that I was part of this great conference. Hopefully next year I will be able to fit OUGF Harmony into my calendar and so should you.

"I am Finnish and this is the end" - Heli

 

07 April 2014

Current_Schema and the Data Dictionary

Being a huge fan of Logger, the PL/SQL logging utility, I really wanted this be to included in the project that I'm currently working on. So I downloaded it (link at the bottom of this blog) and included it in our deployment scripts. Done.... at least I thought so, but of course this wasn't the case.

The regular install script for Logger looks something like the following (parts removed and table names are changed):

set verify off serveroutput on

begin
   execute immediate 'create table new_table (x int)';
end;
/

prompt ****
prompt *** Query user_tab_columns
prompt ****
declare
    l_nullable varchar2(32767);
begin
   select nullable
    into l_nullable
    from user_tab_columns
   where table_name = 'NEW_TABLE'
     and column_name = 'X'
   ;
   dbms_output.put_line('Found the NEW_TABLE, do some stuff to it');
end;
/
The first lines make sure that the user is not prompted for input and to see output from DBMS_OUTPUT.
The "Create table" is done using dynamic SQL after which some more action is taken. To determine which action needs to be taken is handled in the PL/SQL block - now simply replace by some DBMS_OUTPUT. In this PL/SQL block the datadictionary view USER_TAB_COLUMNS is queried.
This install script is usually run in the schema where the objects need to be installed.

This time, however, the install script needed to be adjusted for the DBA to run the script. As the DBA didh't want to log into different schema's during the complete install, the files needed to include the following at the top of the scripts:

alter session set current_schema = &1;
What this does is best explained in the Oracle documentation:
This setting offers a convenient way to perform operations on objects in a schema other than that of the current user without having to qualify the objects with the schema name. This setting changes the current schema, but it does not change the session user or the current user, nor does it give the session user any additional system or object privileges for the session.
The install script was now called (by the DBA) as follows:
@install.sql SCOTT

Adding the ALTER SESSION command at the top of the install script produced this output

Session altered.


PL/SQL procedure successfully completed.

****
*** Query user_tab_columns
****
declare
*
ERROR at line 1:
ORA-01403: no data found
ORA-06512: at line 4
I would have expected that changing the CURRENT_SCHEMA would also query the USER_TAB_COLUMNS related to the schema that we changed into. This is not the case.

New install script

The install script needed to be adjusted to query the DBA_TAB_COLUMNS instead of the USER_TAB_COLUMNS:

set verify off serveroutput on

alter session set current_schema = &1;

begin
   execute immediate 'create table new_table (x int)';
end;
/

prompt ****
prompt *** Query dba_tab_columns
prompt ****
declare
    l_nullable varchar2(32767);
begin
   select nullable
    into l_nullable
    from dba_tab_columns
   where owner = upper ('&1')
     and table_name = 'NEW_TABLE'
     and rownum = 1;
   dbms_output.put_line('Found the NEW_TABLE, do some stuff to it');
end;
/
Now the install script works as desired:
****
*** Query dba_tab_columns
****
Found the NEW_TABLE

PL/SQL procedure successfully completed.

Difference between SESSION_USER and CURRENT_SCHEMA

SQL> col session_user format a35
SQL> col current_schema format a35
SQL> select sys_context ('userenv', 'session_user') session_user
  2        ,sys_context ('userenv', 'current_schema') current_schema
  3    from dual
  4  /

SESSION_USER                        CURRENT_SCHEMA
----------------------------------- -----------------------------------
SYS                                 SCOTT

Not so strange

Knowing what you know after reading the above, the following is not so strange anymore:

SQL> desc t
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------
 STR                                                VARCHAR2(25)


SQL> select column_name
  2        ,data_type
  3    from user_tab_columns
  4   where table_name = 'T'
  5  /

COLUMN_NAME                    DATA_TYPE
------------------------------ -------------------------
ID                             NUMBER

These scripts were run on the following version:

SQL> select *
  2    from v$version
  3  /

BANNER
-------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - Production
PL/SQL Release 11.2.0.2.0 - Production
CORE    11.2.0.2.0      Production
TNS for Linux: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production

Links