Release Notes

Version 2.1.2

RELEASE DATE: Apr 12TH, 2017

Optimizations for group by/aggregation queries on top of UNION and bug fixes mostly related to QLIK function.

Improvements

JD-4144 Support cube generate execution hosts "auto" to automatically set it to local host and known server port
JD-4181 Optimization: Push down GROUP BY and/or AGGREGATIONS to flat sub-query with and without UNION ALL

Bug Fixes

JD-3424 Service command report stopped jethro services as "already running" if instance name contained in another running jethro service
JD-3596 Query execution hangs following dynamic joined table switch on complex join with IN sub-query
JD-3814 Query return error following cubes match failure due to duplicated measures
JD-4009 Garbage collector false reports deletion of keys files
JD-4072 Call to SHOW FULL COLUMNS fails with null deredrench error when encountering invalid views
JD-4083 Query find a cube with duplicate measures but the cube is not used by planner (adaptive entry not matched)
JD-4118 Cube match internal error fails query instead of skipping cube match
JD-4142 Qlik set analysis with assume referential integrity using TOTAL and join fails with error 
JD-4147
Qlik set analysis filter expression fail to match literal with upper case letters (since version 2.1.0)
JD-4179 Adaptive cache with QLIK function - Query returns wrong results from adaptive cache in cases query contains duplicate measures
JD-4183 Throttling in high concurrency scenarios and queries delayed when schema altering command are executed
JD-4212 Query with wrong QLIK function syntax cause server failure 
JD-4213
 Qlik query fails with "position" exception when duplicate measures encountered


Version 2.1.1

RELEASE DATE: MAR 20TH, 2017

Use tcmalloc allocator and bug fixes.

Improvements

JD-3558 use google allocator – tcmalloc

Bug Fixes

JD-4074 Following schema changes all queries are resent to cube generation queue. This is redundant and cause unnecessary cubes generation operations
JD-4021 Maint service minor memory leak created following JD-3852 fix in 2.1.0
JD-4109 Maint service major memory leak related to adaptive rank file read/update


Version 2.1.0

RELEASE DATE: MAR 3RD, 2017

Support cubes regenerate following tables, cubes operational improvements and bug fixes.

New Features

JD-3851 Regenerate cubes following table overwrite

Improvements

JD-3852 Accelerate adaptive space cleanup by deleting invalid entries before adaptive space is full
JD-3854 Cube related logs message improvements

Bug Fixes

JD-3847 Missing system table load event with current status for failed loads
JD-3853 Invalid de-ser debug message filling log
JD-3921 JDBC\ODBC – Query failed. Reason: Server failed to find session
JD-3928 Cube update cause cube generation server failure
JD-3929 Query with union all and that include repeated literals on one side of the union causes server failure
JD-3944 Server fails to deserialize huge cubes with count distinct fetching in on demand mode
JD-3967 Server failed to find session after 2nd call to SHOW ACTIVE QUERIES
JD-4006 Query with CURRENT_TIMESTAMP sent from Tableau fails with syntax error
JD-4017 when the .dat file is removed from adaptive entry, the size of the entry should be zero in show cubes command


Version 2.0.5

RELEASE DATE: JAN 31TH, 2017

Including LDAP authentication (beta), optimization for union queries, additional minor improvements, bug fixes and

New Features

JD-3632 LDAP Authentication (beta)

Improvements

JD-2855 Rewrite: push where predicate to union all
JD-3668 Add timestamps to dmesg output
JD-3669 Add show version to collect logs sql script
JD-3698 Auto cubes: allow cube to match agg(if…) expressions which are decomposed when there is more than one column in the condition
JD-3742 Send earlier status from JethroLoader to the system table
JD-3813 Add SHOW CUBES to collect logs
JD-3817 Add monitor.log to collect logs
JD-3843 Add crontab list to collect logs

Bug Fixes

JD-3670 Select with count(distinct) over cube can cause server failure
JD-3687 Cube matching stops following exception at reading schema
JD-3704 Mishandling of null values in the JDBC driver
JD-3741 Generate cubes – ORDER BY mistakenly added to the cube expression
JD-3750 Cube generation queue is limited to 1000
JD-3751 Update cubes – ID 0.0.0 is mistakenly printed to log as an indication for every query execution
JD-3752 Update Cubes – Update cube should be executed for each key (the cube) instead of executing the original query
JD-3753 Update Cubes – When changing entry type to CUBE we original query SQL is not updated
JD-3754 CASE with aggregation function expression in condition/results fail with “function not found” exception
JD-3762 Jethro ODBC, wrong vendor name.
JD-3771 JethroAdmin create-instance’ doesn’t start the instance automatically when performed using ssh
JD-3785 JethroMaint service failure with: “Too many open files…” messages
JD-3787 Push where wrong write when NOT operator is above AND – possbile occuracy issue
JD-3789 Select with count(distinct) using per partition UKI optimization can cause server failure
JD-3799 Upgrade ODBC SDK
JD-3805 ODBC minor fixes
JD-3806 Tableau TDC – jethro requires explicit cast of null literals
JD-3831 Loader – $LOAD_START load wrong values
JD-3836 Adaptive is not copied to local cache (adaptive fetch to local cache is enabled as global parameter)
JD-3840 Adjust Jethro ODBC to support known escape sequence for outer join {oj ….}
JD-3849 Significant memory leak overtime when enabling cubes


Version 2.0.4

RELEASE DATE: JAN 4TH, 2017

Instruction for upgrade existing instances to 2.0.4

Version 2.0.4 enables system tables events creation by default. The mandatory requirement for system tables is to have loads scheduler services (JethroLoadsScheduler) run automatically. The service must be started and enabled to run automatically as follows:

1. Edit the service.ini file and set the third parameter for all active instances to yes
   > vi $JETHRO_HOME/instances/services.ini
   –> {instance-name}:port:yes:yes:yes

2. Start the Jethro loads scheduler service:
   > service jethro start {instance-name} loadscheduler

New Features

JD-3593 Enable system table events creation by default

Improvements

JD-2784 Flatten query write: remove sub-queries and merge it into the upper query to reduce multi-level queries overheads and avoid where on external queries (optimization not enabled by default)
JD-3591 Count distinct optimization for high cardinality columns: evaluate count distinct per partition to reduce memory footprint and enable using fast bitmap based evaluation
JD-3697 Show cubes – add status column

Bug Fixes

JD-2873 Push where – not performed if contains IN clause
JD-3173 Fix issues with local cache statistics
JD-3129 Push where – Potential server failure following applying push where optimization
JD-3607 Slow query execution with count distinct
JD-3677 Push where – serve failure following partial push where (one of the where expressions is IN expression)
JD-3691 Accuracy issue: when where expression contains NOT IN and join index defined on the NOT IN column
JD-3700 Adaptive cache – adaptive cache results not being invalidated after dim table append


Version 2.0.3

RELEASE DATE: DEC 16TH, 2016

Bugs fixes and improvements.

Issues list:

Improvements

JD-3329 Utility to reconstruct column index from column data
JD-3622 Expand cubes with where support to allow complex WHERE clauses
JD-3662 Generate cube – additional informative message in jethroserver.log when cube is not generated
JD-3675 Enable hi performance count distinct execution mode via count distinct container for CSN128 and Value Hash group by plans

Bug Fixes

JD-3297 Query with HAVING is not finding a cube match
JD-3404 Cube not found by Cube Generator while cube gen query sent to server find a cube
JD-3597 Query hangs – join with right table is sub-query that returns very big result set
JD-3617 Server failure due to query retry as a result of adaptive fingerprint mismatch exception
JD-3649 Memory leak on cubes generation
JD-3654 Adaptive incremental mode not active if new partition was added
JD-3664 Local cache does not update itself with new events (since 2.0.2)
JD-3681 Memory leak at count distinct queries using count distinct container


Version 2.0.2

RELEASE DATE: NOV 25TH, 2016

Bugs fixes and minor improvements.

Issues list:

Improvements

JD-3401 ODBC: Enhancements for copy TDC file in ODBC installation

Bug Fixes

JD-3458 Jethro server failure during active load while frequent queries sent to the system table (adaptive problem)
JD-3606 Maint service fail and stop due to problem in garbage collection
JD-3608 Maint service fail to initialize in case it fails to commit session configuration
JD-3619 ODBC: fix issue that cause Tableau to fail when disconnect is called without connect


Version 2.0.1

RELEASE DATE: NOV 17TH, 2016

Fixes for 2.0.0 known limitations/bugs.

Issues list:

Improvements

JD-3574 Stop/start of cubes generation/update via set command (no maint restart required)

Bug Fixes

JD-3416 Cubes with avg(if) are not matched
JD-3585 Auto cubes – read from cube accuracy issue when two different aggregations encapsulate an identical CASE/IF expression
JD-3586 SHARED.WHERE WARN Release shared object cannot found a relevant object in map
JD-3587 Auto cubes – read from cube accuracy issue when AVG aggregation encapsulate CASE/IF expression
JD-3588 JethroClient does not exit gracfully after failure (exception handling issue)


Version 2.0.0

RELEASE DATE: NOV 15TH, 2016

Jethro 2.0 introduces auto micro cubes.

Jethro auto micro cubes brings the great performance value of pre-aggregated cubes, combined with the ability to automatically generate and transparently maintain those cube.
Cubes can be useful to accelerate performance for certain set of queries, mainly queries that are not highly filter and would normally scan large parts of the data set. Jethro optimizer combines cubes with indexing and caching technology to ensure interactive queries response time on wide range of use cases.

Issues list:

New Features

JD-2590 Auto Micro Cubes – Match and rewrite dynamic aggregations
JD-2609 Adaptive incremental – Expend support for additional queries types (CSH Hash)
JD-2786 System Tables
JD-2788 System Table for loader events
JD-2919 Auto Micro Cubes – Dynamic aggregation of count distinct queries
JD-2921 Auto Micro Cubes – Support hierarchy levels for timestamp columns
JD-2937 Auto Micro Cubes – Auto generate dynamic aggregations from incoming queries
JD-3110 Auto Micro Cubes – Manual cubes generation via GENERATE CUBES
JD-3111 Auto Micro Cubes – Support cube with where
JD-3112 Auto Micro Cubes – Find match for aggregators that has internal case expression
JD-3118 Auto Micro Cubes – Match count distinct aggregation queries without group by to cubes
JD-3411 Auto Micro Cubes – Allow aggregation queries without group by and with where to generate and match cubes
JD-3345 Auto Micro Cubes – Generate and match cubes for queries without measures
JD-3348 Auto Micro Cubes – SHOW CUBES and related cubes management commands

Improvements

JD-2838 Enable fetch and read adaptive cache files to/from local cache
JD-2839 Add section separators to the jethro loader report file
JD-2878 Add watchdog for JethroMaint
JD-2918 Optimized count distinct for CASE/IF in distinct aggregator
JD-2938 New instances parameters should be set with Yes as default in Service.ini
JD-3043 Count Distinct Optimization: Allow push group by functions and joins to upper query with count distinct using distinct values bitmaps and support adaptive incremental for count distinct
JD-3085 Improve adaptive write efficiency via large write chunks
JD-3183 Adaptive cache read: split each column read according to the size of current TupleSet parameter
JD-3210 Update incremental cubes by maint
JD-3285 Show active queries – add new option to status column: caching

Bug Fixes

JD-2204 Server failure when query is VERY long and includes text after “;”
JD-3137 Memory leak – related to IN with sub-query in outer query
JD-3142 Memory leak of Global UKI
JD-3363 Possible deadlock/leak in buckets merge
JD-3370 Join index not used when where clause contains OR not null
JD-3379 Server failure when query reads all adaptive columns into single buffer
JD-3467 Query failed with error – limit issue
JD-3507 Jethro server failed to execute query with IN literal left operand
JD-3514 Jethro Server failure – query hit partition holder internal error


Version 1.6.11

RELEASE DATE: NOV 9TH, 2016

Bug Fixes

JD-3461 Fix for JethroAdmin list-storage-instances
JD-3544 Query with UNION ALL as subquery stored in adaptive cache failed to update on data load/update
JD-3425 Loader doesn’t reject tokens with blank spaces loaded into numeric columns
JD-3524 Fix server failure in case there is filter with IN expression and a query on the right side of IN has filter of its own
JD-3438 Fix local-conf typos


Version 1.6.10

RELEASE DATE: OCT 12TH, 2016

Improvements

JD-2910 Copy TDC file to the right place upon ODBC installation
JD-3068 SHOW PARAM: present changed parameters modification level including set global and support UNSET command
JD-3239 Add driver version (odbc, jdbc, JethroClient) to JethroServer log
JD-3277 Improve loader performance (streamer)
JD-3282 Collect logs should now collect revisions of schema.xml
JD-3290 Update status codes returned from the ODBC
JD-3298 Loader performance improvement – low cardinality keys
JD-3330 SHOW TABLE COLUMN: reduce memory consumption when there are multiple high cardinality columns
JD-3389 Collect-logs – add instance name and date to log files and improve error handling

Bug Fixes

JD-1506 IN/NULL issue: NOT IN with NULL value in IN list should not return the NULL value
JD-2690 IN/NULL issue: IN expression with list of literal doesn’t ignore NULL in the list
JD-3108 Adaptive cache error in log: “failed to get schema object”. Not an error – should be trace message
JD-3318 ORDER BY return wrong order when sorting positive and negative numbers
JD-3319 UNION ALL – duplicated columns
JD-3320 Session and global parameter files potentially can be corrupted if SET commands issued concurrently
JD-3336 Inconsistent EFS list operation cause wrong read of metadata and valid columns removed by GC. Fix version file validation check
JD-3351 UNION ALL with duplicate literals – duplicated columns when reading from adaptive
JD-3364 Partition size limitation of 4B rows due to file identifier rowID wrap around
JD-3376 Collect logs password is not provided and no appropriate error is issued
JD-3378 Server failure for queries with duplicate aggregators when reading from adaptive in incremental scenario
JD-3381 odbc 32 on windows 7: Cannot create a DSN


Version 1.6.9

RELEASE DATE: SEP 6TH, 2016

New Features

JD-3092 KILL QUERY – interactive kill of long running query

Improvements

JD-3175 Add JDBC support for getTimestamp(calendar) and item MergeColumnHeaderCells methods
JD-3209 support ORDER BY degrees (ordinal number)
JD-3215 Optimization for string functions in where: add optimization for IN (func(col) IN (…))
JD-3280 Support queries selecting literals without FROM clause

Bug Fixes

JD-2663 Wrong result when using cast as “float” with quotes
JD-2769 JethroClient issue error when run in the background
JD-3091 Query cancel does not terminate certain queries
JD-3124 Lock files are not removed
JD-3142 Memory leak of Global UKI
JD-3205 Wrong results: where optimization for string function return treat NULL return value as true
JD-3267 DDL commands fail after drop of dim table (drop table, show tables extended, create join index,…)
JD-3272 Loader and Maint fail following upgrade from 1.4.6 to 1.6.8 (fix leftover files handling)
JD-3276 Server failure during query following partition overwrite


Version 1.6.8

RELEASE DATE: AUG 11TH, 2016

New Features

JD-2787 Scheduled Loads (beta)

Improvements

JD-2930 Add command JethroAdmin list-storage-instances
JD-3016 JethroAdmin collect-logs support password entry
JD-3050 Limit memory factor to avoid huge memory consumption for base parameters
JD-3097 Set instance name max size to 32 characters
JD-3198 Allow Jethro installing with a user other then jethro

Bug Fixes

JD-2998 Server failure if order by column appear in group by inside a function
JD-3011 collect logs does not check instance name is valid – reports success
JD-3071 Illegal query combining Distinct and Order by causes server failure. Should issue an error.
JD-3093 Rare race condition on high concurrency causes all queries to wait on query queuing until time-out exceeded
JD-3114 ODBC: return true max column length for string
JD-3138 Fix threads counter to include all threads including top query thread


Version 1.6.7

RELEASE DATE: JUL 20TH, 2016

Bug Fixes

JD-3104 JethroMaint service stops after DROP PARTITION command is issued. This problem existing only in version 1.6.6 and affect only tables loaded only with version 1.6.6 Loader.


Version 1.6.6

RELEASE DATE: JUL 15TH, 2016

Improvements

JD-2530 Move Loader report log, profiles and rejects files for specific log folder
JD-3020 Allow setting loader job name and include it in loader report file name
JD-3048 TDVT: Support required function: Right(string, int)
JD-3059 Optimize query with group by and count distinct on the same column
JD-3060 Reduce PLANNER time for queries with very high cardinality (improve global UKI access)
JD-3063 Optimize Join performance: dynamically switch left/right tables in join node during query execution
JD-3076 Control and reduce memory footprint for instances with large number of string columns (reduce small initial hash size)
JD-3102 Updated Tableau TDC file

Bug Fixes

JD-2373 Jethro server stops/ when local cache is full and stops again to restart
JD-2961 Local cache won’t replace files after cache space is filled / stop fetching new files even when the is available space
JD-3014 Server failure when receiving query over empty table with function on a string under WHERE
JD-3015 Local cache copy files of tables that were dropped
JD-3022 Local cache metadata files potential unsafe commit
JD-3028 TDVT: issues with Locate and Substring functions taking Floor function as argument
JD-3029 TDVT: TIMESTAMPADD fails when getting interval of type Double
JD-3039 TDVT: Change LOG() to compute LN() and add LOG10() to apply to Tableau standards
JD-3066 TDVT: Wrong results – empty set when using cast of string representing real number to integer
JD-3074 ODBC does not signal exception when it occurs during accumulating results from server
JD-3080 Query on empty table using string functions fail with “Illegal construction of TupleSet. Size = 0” error
JD-3081  JDBC problem with getObject() API: Erroneous NULLs appears in a results set rows
JD-3082 SHOW LOCAL CACHE cause server failure when local cache path not defined
JD-3084 Query with UNION and COUNT DISTINCT fails with “unsupported expression type: 22”
JD-3086 Server failure on rare scenario when executing query on empty tables
JD-3095 SHOW LOCAL CACHE – shows inaccurate statistics
JD-3096 lock ignored as stale though was not actually stale – increase stale time out


Version 1.6.5

RELEASE DATE: JUN 27TH, 2016

Improvements

JD-2464 Ignore range in WHERE that includes all the values
JD-2962 SHOW LOCAL CACHE – make more usable: show cache/touched/storage information per table
JD-2987 Support round with precision
JD-3010 Improve fetch statistics logs
JD-3032 Loader streamer performance: improve rejects handling performance

Bug Fixes

JD-1734 Query CANCEL: fix scenario were query is not stopped when cancelled
JD-2654 Corrupted lock file issue causing locking to fail consistently
JD-2704 Query file if target list in select clause doesn’t include at least one column (for queries with having clause)
JD-2706 Case function can’t handle comparison between BIGINT and DOUBLE
JD-2760 Crash in SHOW LOCAL CACHE EXTENDED
JD-2761 Local cache disk full cause server crash on init
JD-2953 JethroServer – configuration values are transformed to lower case (problem for path values with upper case)
JD-2965 ODBC client: fix scenarios were query is not stopped when cancelled from ODBC
JD-2986 abs function not found
JD-2989 ODBC failed to handle group by boolean (fixed in ODBC 1.7)
JD-2994 ODBC 32bit doesn’t use TDC
JD-3004 Tmp files not deleted from instance-storage/tmp
JD-3024 Query with sub-query that contain tables from the outer query fail (server failure). Related to push where
JD-3053 Query with count distinct on QLIK function fail (server failure)


Version 1.6.4

RELEASE DATE: JUN 9TH, 2016

Improvements

JD-2972 Trim CASE/IF branches with false conditions
JD-2973 JethroAdmin collect log: force instance name and create collect log file name with date and time
JD-2985 Rewrite: additional clean-up for simple math operations in SUM aggregate
JD-2988 Improve adaptive read efficiency
JD-2995 Aggregation optimization: clean cast as date for min/max aggregations to achieve index based aggregation

Bug Fixes

JD-1552 Server crash on group by query following drop partition or failed load if number of unique strings loaded gt number of unique values in the string columns
JD-2933 Join with Union all returns bad results when adaptive is turned on. (result of col1 is duplicated to col2)
JD-2994 Tableau with ODBC 32bit doesn’t use TDC
JD-2996 Wrong results: filter on right table column from left join is ignored following push where optimization (JD-2785)
JD-3000 Server crash on Query that cast BIGINT to STRING
JD-3008 Adaptive read issue
JD-3009 Adaptive errors on WHERE IN query


Version 1.6.3

RELEASE DATE: MAY 26TH, 2016

New Features

JD-2917 Bitmap based GROUP BY COUNT DISTINCT. X5-X100 performance acceleration for many scenarios
JD-2928 Support SQL_TSI_FRAC_SECOND for timestampadd/diff and atan2 (Tableau requirements)

Improvements

JD-2619 Print modified configuration parameters at jethroserver.log start
JD-2840 Write adaptive indexes maps asynchronously. This allow keeping larger adaptive index maps.
JD-2943 Remove star transformation limitation for fact and dim
JD-2949 Fix fetch and execution tuple set sizes calculation to allow more parallel processing
JD-2951 Improve performance for where with functions on string columns

Bug Fixes

JD-2914 Adaptive write for large tuple set is exteriorly slow – need to optimize inefficient implementation
JD-2924 Issue with global configuration. Server fails to init after setting global config
JD-2940 Query failure: UNION ALL inside IN sub-query
JD-2957 TIMESTAMPDIFF – Accuracy issue when swapping the literals inside the function
JD-2958 Wrong expression ID allocation when rewriting aggregation (sum(-C1)) (was Internal Error)
JD-2968 TIMESTAMPADD – microsecond – Accuracy issue when using negative number in TIMESTAMPADD
JD-2975 Remove show local cache from collect-logs (pending show local cache fix)
JD-2980 Shared warn potential issue: release shared object cannot found a relevant object in map


Version 1.6.2

RELEASE DATE: MAY 6TH, 2016

New Features

JD-2669 Loader support hex value input to INT and BIGINT data type

Improvements

JD-2785 Rewrite: push where clause from outer query to inner query (optimization for tableau custom query)
JD-2865 Range Index – rewrite with IN to compute OR of index maps concurrently and to add to adaptive index
JD-2880 Functional Index – rewrite with IN to compute OR of index maps concurrently to add to adaptive index
JD-2903 Join performance: avoid copy of qualification columns during join execution
JD-2913 Performance – optimize GROUP BY over YEAR/MONTH/DAY functions with CAST to DATE

Bug Fixes

JD-2817 Various issues with transform of expression to SQL
JD-2899 Range index – block range index creation on data types other the INT and BIGINT
JD-2915 “CURRENT_TIMESTAMP” with quotes is not identified
JD-2916 Server crash on partition pruning while running now() on hours


Version 1.6.1

RELEASE DATE: APRIL 27TH, 2016

Improvements

JD-2867 Enable adaptive read from sub-queries
JD-2893 Performance – Force to push JOIN if there is where on the DIM and fact doesn’t include aggregations

Bug Fixes

JD-2895 Join index not used with QLIK function creating specific filter
JD-2901 Push join doesn’t push because wrong calculation of potential groups
JD-2904 Truncate table fails after load overwrite finish unsuccessfully
JD-2907 Fail to read from view (adaptive read fail) – getting CHECK failed: (index) lt (size())
JD-2911 Adaptive cache – sub-query results written to adaptive cache if sub-query is a view (should be disabled)


Version 1.6.0

RELEASE DATE: APRIL 20TH, 2016

New Features

JD-2811 Range indexes – optimize range filtering using indexes on ranges of values
JD-2847 Shared Where – cache where results for reuse by other concurrent queries that issues the same where clause

Improvements

JD-2662 Rewrite: Basic cleanup of functions in aggregators
JD-2863 Set ulimit by Installer to support high concurrency
JD-2869 Allow drop of multiple partitions in one operation
JD-2886 Performance – CAST to DATE in GB is not optimized

Bug Fixes

JD-2780 Partitions not pruned if same table appear more then once in the query
JD-2834 Query with left join return empty set when right table is empty
JD-2846 Local cache auto refresh doesn’t fetch new data if server was running before load start and schema was changed
JD-2848 Query with star transformation fails (issue with join remove)
JD-2854 Join column display order – on left join left table columns should be before right columns
JD-2859 Fetch dispatcher – constructor thread leak
JD-2861 JethroData::Lock deadlocks under intensive use
JD-2871 Fetch from column after adding it with add column fails
JD-2875 Pre-query operations are not measured in query total time
JD-2888 JethroLoader – first block size in each partition is less then blocksize


Version 1.5.2

RELEASE DATE: APRIL 4TH, 2016

New Features

JD-2745 Loader: support single digit TIMESTAMP format for month / day / hour / minute / sec
JD-2791 Show version command

Improvements

JD-2770 Loader: Report of success or failure to console
JD-2800 Optimize filter where: avoid processing blocks skipped for push down predict scenarios
JD-2807 Optimize filter where: remove HAVING node from filter where
JD-2819 Change default fetch tuple set size configuration to reduce memory allocation during fetch
JD-2823 set CAP_FAST_METADATA in TDC

Bug Fixes

JD-2461 Wrong results – rare potential error in adaptive delta: if not all partitions updates or if partitions are dropped and added
JD-2675 Cast to VARCHAR fails
JD-2759 Tableau doesnt uses CASE to convert boolean to integer – ODBC failes (due to JD-2726)
JD-2768 Empty set returned from adaptive cache following unexpected client disconnection
JD-2774 Performance: allow group by push function rewrite with count distinct aggregator if column in function already appear in group by
JD-2779 PushDown group by optimization fails on group by functions
JD-2814 Server failure on query with IN expression within function (CASE) inside the WHERE clause
JD-2822 JethroServer does not report fatal initialization errors to console (e.g. port busy)
JD-2825 Query failure (aggregation in CASE function)


Version 1.5.1

RELEASE DATE: MARCH 21TH, 2016

New Features

JD-2551 Add new functions: round(), sign(), sqrt(), space(), ascii(), tan(), cot(), atan(), sin(), asin(), cos(), acos(), degrees(), radians(), pi(), exp(), ln(), log(), replace()

Improvements

JD-2589 Support Redhat/Centos 7.2 (resolve service start/stop issue)
JD-2607 Optimization rewrite: Push multiple joins in group by to upper query
JD-2720 Add support for millisecond period in functions timestampadd() and timestampdiff()

Bug Fixes

JD-2783 Server failure on query IN clause inside aggregator
JD-2790 Results fix for queries with cast(func(..)) target list
JD-2796 Join Index – set index block size in relation to data block size
JD-2798 Now() function return current date without hours/minutes/seconds


Version 1.5.0

RELEASE DATE: MARCH 14TH, 2016

New Features

JD-2757 Support char() function

Improvements

JD-2378 Significantly improve high frequency small load support: data model block sizes is unaffected by load size

Bug Fixes

JD-2740 Union all fails or wrong results with order by or limit clauses
JD-2781 Query stuck when all partitions are pruned and select target list count() aggreagtor


Version 1.4.6

RELEASE DATE: MARCH 13TH, 2016

Improvements

JD-2754 Loader performance improvement – create global UKI only for updated partitions when finalizing


Version 1.4.5

RELEASE DATE: MARCH 8TH, 2016

Improvements

JD-2615 Decrease CROSS JOIN memory usage
JD-2765 Optimization: allow trying star transformation without where optimization for STRING quals if number of keys is small

Bug Fixes

JD-2489 Error when Tableau is connecting to an EMPTY Jethro instance
JD-2531 JDBC causes java.net.SocketException
JD-2575 Loader option LINE TERMINATED BY doesn’t work. Option disabled
JD-2598 CSN node execution time is higher than expected
JD-2660 JDBC – bad error message when no instance is specified in url
JD-2711 Server error: Local cache manager exit
JD-2721 Loader report – improperly printed reject file name
JD-2729 Double slashes in local cache path will prevent auto filling cache
JD-2751 Default value of query.optimizer.grouping.hash.base.size (10m) is too big and can add few seconds to query time due to large mem allocation
JD-2753 local-conf does NOT recognize parameters with capitalized letters and does not report them as wrong
JD-2764 Functional index for date/time optimization doesn’t work when timestamp literal is left of the column identifier
JD-2771 Exception on failure creating new thread not handled properly
JD-2772 Load table in overwrite mode may create wrong join indexes if wrong (non-default) block size settings are used


Version 1.4.4

RELEASE DATE: MARCH 1ST, 2016

Improvements

JD-2604 Add SQL Statistics to JethroAdmin collect-logs

Bug Fixes

JD-2760 crash in SHOW LOCAL CACHE


Version 1.4.3

RELEASE DATE: FEBRUARY 29ND, 2016

New Features

JD-2723 Support char(int or float) – TABLEAU only
JD-2744 Support curdate() function

Improvements

JD-2034 Improve fetch performance: decompression performance analysis and improve
JD-2719 Handle currently ODBC cast syntax for DATE “{ D …} and/or support cast to DATE with DATE keyword
JD-2726 Get from Tableau standard expressions instead of CASE WHEN boolean expression to gain index performance boost
JD-2735 Enable returning table and column identifiers as uppercase strings (configuration based)
JD-2746 Queries with now and date can be saved in adaptive cache since values are computed and saved as literals

Bug Fixes

JD-2734 concat doesn’t support concat of ‘/n’ sent from tableau
JD-2742 Query grouped by function with 2 or more identical aggregators fail
JD-2743 Query grouped by function and literal fails


Version 1.4.2

RELEASE DATE: FEBRUARY 22ND, 2016

Improvements

JD-2684 Support IFNULL function in Jethro SQL
JD-2701 Change NULLIFEQ() name to NULLIF()
JD-2709 Local cache refresh – use event update instead of file open

Bug Fixes

JD-2667 Multiple UNION ALL in one query fail
JD-2673 UNION ALL Query stuck while UNION ALL and JOIN combine on the same expression
JD-2676 UNION ALL Alias is not presented in query results
JD-2686 Unable to use literals/NULL as target column in VIEW
JD-2714 Wrong results related to adaptive sub-query in a specific scenario


Version 1.4.1

RELEASE DATE: FEBRUARY 15TH, 2016

Improvements

JD-2682 Allow join indexes to be create over dimension with no primary key
JD-2670 Loader format function unix_timestamp support fractional milliseconds

Bug Fixes

JD-2612 Query with group by count distinct take very long time
JD-2635 Issues with the distributed lock mechanism
JD-2664 Exception on partition pruning
JD-2672 Local cache refresh new files fails on hdfs
JD-2689 JethroServer creates too many threads causing queries fail
JD-2692 Performance: FI for range expressions (between) not always used
JD-2699 Query/server fail when using function over IN column: F(C1) IN (…)


Version 1.4.0

RELEASE DATE: FEBRUARY 2ND, 2016

Backward Compatibility

Data model conversion from previous versions is required.
Before starting service run: > Jethro U {instance-name} T

New Features

JD-2349 Support UNION ALL

Improvements

JD-2533 Column block encapsulation – store column block metadata and compressed value in a single file and avoid file append

Bug Fixes

JD-2522 Empty set result instead of exception that was raised – not consistent
JD-2637 Remove a warning message in partition pruning
JD-2645 Filter with “IN” does not work when the filter column is a partition key
JD-2653 Maint keeps life-long lock and old lock versions are not removed.
JD-2655 Query with quotes around the datatype in CAST fail
JD-2657 Sub-query adaptive causes simple query with sum and group by to fail. Return unclear error: CHECK failed: (index)  lt (size())


Version 1.3.5

RELEASE DATE: JANUARY 28TH, 2016

Improvements

JD-2634 Optimization for group by function queries: push groups expression to outer queries

Bug Fixes

JD-2595 Slow performance when large joined table key is non-unique and there are multiple rows for each key
JD-2620 JethroServer crash after create/drop tables due to local cache pre-fetch
JD-2633 Performance degradation for distinct queries after after upgrade to 1.3.4 due to change in assume referential integrity
JD-2636 Various issues with assume.referential.integrity version 2


Version 1.3.4

RELEASE DATE: JANUARY 21TH, 2016

Improvements

JD-2491 Optimization: Push Join after GB to upper query (single join)
JD-2525 Assume referential integrity – version 2
JD-2538 Enable adaptive cache for sub-query results
JD-2582 SHOW ALL TABLES/PARTITIONS/COLUMNS
JD-2613 Reduce hash size factor of join node hash

Bug Fixes

JD-2541 JethroLoader Crash when the path to the local cache directory in local-conf.ini is wrong
JD-2577 Service jethro restart – try to bring up a new Maint before the old one went down completely.
JD-2597 Wrong results when using CAST over aggregator in queries optimized with push joins to upper query
JD-2611 Wrong results: GB with CSN 128bit returns
JD-2614 Complex query with cross join and string column crash


Version 1.3.3

RELEASE DATE: DECEMBER 30TH, 2015

Bug Fixes

JD-2584 Block drop partition column with alter table drop column
JD-2593 Loader can crash we using local cache in load


Version 1.3.2

RELEASE DATE: DECEMBER 28TH, 2015

Improvements

JD-2496 Allow more than one <=> operator in JOIN qualification

Bug Fixes

JD-2583 ADD COLUMN fails when table has partitions
JD-2585 Wrong results when using offset gt number of rows in result set


Version 1.3.1

RELEASE DATE: DECEMBER 18TH, 2015

New Features

JD-2555 Support OFFSET clause in LIMIT clause

Improvements

JD-2192 Improve Show Local Cache information

Bug Fixes

JD-2542 Partition pruning – fix minor memory leak
JD-2554 Crash on partition pruning if table appears in more than one sub-query
JD-2572 Loader memory leak – keys pool doesn’t release temp buffer


Version 1.3.0

RELEASE DATE: DECEMBER 12TH, 2015

Backward Compatibility

Data model conversion from previous versions is required.
Before starting service run: > Jethro U {instance-name} K

New Features

JD-1526 Partitions pruning
JD-2362 Support Tableau Date data type usage
JD-2515 Support SQL functions “TIMESTAMPADD” and “TIMESTAMPDIFF”

Improvements

JD-2300 Integrate with libhdfs3 (beta mode)
JD-2354 update ODBC convert capabilities
JD-2379 Keep keys (values and indexes) in multiple files (file per block) and support keys recovery
JD-2398 Trivial Distributed locker based on shared file system
JD-2512 QLIK set analysis support: add support for DISTINCT operator with QLIK function
JD-2526 QLIK set analysis: Add ‘TOTAL! ‘ support
JD-2540 Update ODBC capabilities to reflect supported functions

Bug Fixes

JD-2370 CASE – if first then is NULL then CASE may fail due to type mismatch
JD-2486 Incoherent error message when performing a numeric function (e.g. sum) on a STRING column
JD-2501 Client hangs when connection or queries hangs/return after very long time (actual execution time is small)
JD-2502 libhdfs3 – Loader can’t load data files from HDFS
JD-2510 Monitor – No indication when the Jethro server and Maint were stopped
JD-2511 Monitor – No indication for Maint start in the service.log
JD-2520 QLIK set analysis – use left join instead of inner join
JD-2523 Crash happens when IN clause includes NULL value
JD-2524 Loader streamer creates more data blocks then set in config limit
JD-2535 QLIK set analysis – fix cases when original query doesn’t have target column


Version 1.2.1

RELEASE DATE: NOVEMBER 18TH, 2015

Improvements

JD-2503 Loader Usability – error messages
JD-2506 Improve loader memory control: release keys cached buffers if cache is full and allow using local cache during load

Bug Fixes

JD-2077 Loader: various issues with loader errors logging
JD-2307 Fix loader report: add number of rows removed from overwrite scenarios
JD-2418 Fix misleading message when input file is corrupt: Input row is longer than 10m bytes
JD-2455 Change error message in log file when user is running more than one loader on same table.
JD-2492 Loader report: indicate in report file that rows are being rejected and point to rejects file
JD-2508 Missing configuration in loader report file
JD-2509 Loader memory leak for Timestamp column


Version 1.2.0

RELEASE DATE: NOVEMBER 13TH, 2015

New Features

JD-2356 Primary Key
JD-2197 Join indexes
JD-2469 QLIK set analysis functionality in Jethro SQL (Alpha)

Improvements

JD-2470 Join after Dim GB Optimization – basic scenario (distinct values, no aggregators, single join)
JD-2487 Improve assume referential integrity (drop fact from join when possible)

Bug Fixes

JD-2483 Some aggregations are executed via non-optimized path (full scan) instead of index based aggregation
JD-2499 rpm upgrade removes Jethro monitor job
JD-2500 JOIN invalid qualifier crashes


Version 1.1.4

RELEASE DATE: OCTOBER 27TH, 2015

Bug Fixes

JD-2363 Libhdfs messages captured in logs as ERROR even when it’s informational or warning
JD-2460 Error received on group by query using adaptive delta
JD-2465 Loader test and recovery – didn’t work on functional columns


Version 1.1.3

RELEASE DATE: OCTOBER 14TH, 2015

Improvements

JD-2382 Improve performance of GROUP BY queries with datetime functions on groups
JD-2445 Add date/time FI optimizations for YEAR range and IN

Bug Fixes

JD-2361 wrong results for tpch query: Q19
JD-2434 TPCH Q3b query is causing server failure
JD-2429 Server crash during query with where clause that include column<>”
JD-2430 Memory leak in join queries
JD-2436 Loader does not reject lines with timestemp bigger than max timestamp
JD-2444 Loader issue: unix_timestamp format for timestamp in doesn’t work. User gets a reject: “failed to create unix time”
JD-2363 Do not LOG libhdfs stderr capture as ERROR if it is not
JD-2423 Loader exits with positive status when loader reach maximum rejects
JD-2338 JethroAdmin.sh call service stop/start doesn’t support multi versions installation
JD-2432 jethromonitor.sh does not report instances initialization to /var/log/jethro/monitor.log
JD-2186 loader should set number of threads to 1 in the case of 2 core machines


Version 1.1.2

RELEASE DATE: SEPTEMBER 29TH, 2015

New Features

JD-2359 Cross Join support
JD-2367 Null safe equi join with operator <=>

Improvements

JD-2351 String ranges optimization (use indexes by converting to IN)
JD-2400 Rewrite “JOIN … on A=B or A is null and B is null” to use <=> operator

Bug Fixes

JD-2018 Loader crash at create global uki stage for high cardinality column
JD-2309 JethroServer coredump when local-cache.ini is missing
JD-2318 Corrupt persist.xml file
JD-2327 Query with cross join crash the server
JD-2329 Loader does NOT reject INT with spaces: “12 3”
JD-2368 show local cache – crash if issued before first background update
JD-2369 Loader terminates with error reading from file when creating global UKIs
JD-2370 CASE – if first then is NULL then CASE may fail due to type mismatch
JD-2371 Nested queries – crash because semantic error is identified by NS but not captured and is passed to QueryBuilder
JD-2372 nested queries – join tables are not traversed by semantic for validity
JD-2402 Server stuck on execution


Version 1.1.1

RELEASE DATE: AUGUST 23TH, 2015

Improvements

JD-2353 Support assume referential integrity – mode that eliminate all referenced table from join. phase I : support star schema
JD-2355 allow position reference in GB

Bug Fixes

JD-1711 SELECT with sub-query attributes that aren’t supported should be rejected
JD-1826 SELECT accept unsupported options
JD-2316 CASE mishandles NULL type conversion
JD-2322 wrong result: explicit conversion NULL::float returns 0 in CASE statement
JD-2330 ODBC – update and fix connection properties
JD-2347 IN reject correct aggregation function targets in sub-query due to interim data-type
JD-2365 Loader overwrite option removes functional indexes


Version 1.1.0

RELEASE DATE: AUGUST 11TH, 2015

New Features

JD-1045 Function Indexes Phase 1

Improvements

JD-2344 ODBC driver for 32 bit (Windows)
JD-2343 Optimize queries with repeated expressions in target list
JD-2342 Join optimization: add transformation from outer join to inner join to allow star transformation optimization
JD-2341 Improve JOIN optimization – don’t disable star transformation if another outer join exists after inner join
JD-2340 Adaptive caching for IN with Subquery
JD-2199 Shared adaptive cache
JD-2188 Loader: optional per-column NULL representation

Bug Fixes

JD-2337 Fix merge all filter maps before index full scan
JD-2330 ODBC – update and fix connection properties
JD-2328 Fetch threads in filtered where + partitions
JD-2324 wrong results if GB col is also not last join qualification
JD-2298 Fetch SN performance issue
JD-2286 Datediff function performance is slow
JD-2282 missing if() function for TIMESTAMP causes IF stmt to fail and CASE to crash
JD-2239 wrong qualifier identification if multiple ‘=’ used in join ON clause (was: wrong results on JOIN queries with casting)
JD-2238 Query failed with “request execution failed” (iBasis – flat queries)
JD-2232 Query failed without clear error message. Uses unsupported function within CAST
JD-2231 Query failed with “SemanticException – Cross join is not supported – ON clause requires at least one join condition”
JD-2219 Server memory leaks in query execution
JD-2208 Local-cache.ini is not effecting cache refresh
JD-2177 Wrong result for complex expression that use filter where
JD-2122 Service jethro command does not work well in CentOs 7


Version 1.0.2

RELEASE DATE: JUNE 6TH, 2015

New Features

JD-2217 Support MapR distribution
JD-2073 SHOW PLAN – show detailed execution plan (in log)

Bug Fixes

JD-2214 JethroServer crash after getting cancellation from Client (most likely via pentaho)
JD-2213 Query fails error – join between views
JD-2194 Segfault – server fails on a specific query
JD-2191 Query Cancelled by user. resulted in an internal error and query hanging
JD-2187 JerthroAdmin: incorrect error message on attache instance
JD-2185 Problem with environment setup with HDP (Hortonworks) installation
JD-2180 execution counter – hole of 90% of the time – count distinct
JD-2179 JethroServer crashed or stuck on query
JD-2172 Unsupported version message send due to serialization version mismatch doesn’t provide info required to identify to problem
JD-2145 Loader crashed on deadlock
JD-2099 Schema oid usage isn’t efficient and can also cause lock bugs


Version 1.0.1

RELEASE DATE: MAY 10TH, 2015

New Features

JD-1138 ALTER TABLE ADD/DROP COLUMNS

Improvements

JD-2168 refactor adaptive rank calculation
JD-2005 Dynamic refresh of local cache based on blocks access frequency

Bugs

JD-2178 JethroODBC: ODBC driver should disable catalog support
JD-2176 Jethro initialization failed on test server
JD-2173 when adaptive cache deletes a query, it writes many junk lines in jethro.log
JD-2171 count(*) on columns from sub-query fails ” ‘Scalar’ node build failed”
JD-2117 Adaptive query cache – use FULL if can’t use INCREMENTAL
JD-1950 Query with IN returned wrong result on branch ‘JD-1947_non_opt_build’