Online URL Encoder

Trying to build an external hive table from .gz log files present in HDFS in hive.After running the query:

CREATE EXTERNAL TABLE table_name(att1 STRING,att3 STRING,att4 STRING,att5 STRING) row format serde "org.openx.data.jsonserde.JsonSerDe" with serdeproperties ("ignore.malformed.json"="true") STORED AS TEXTFILE LOCATION 'hdfs:////hdfs_location/';

when i run

   select count(*) from table_name;

it fails by giving below stack trace:

TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1534417036833_0016_1_00_000054_2:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.EOFException: Unexpected end of input stream at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.EOFException: Unexpected end of input stream at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:74) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185) ... 14 more

I tried looking into the content by validating the json.Ran a ruby script to check for the contents:

require "zlib"require "json"path = "/home/test_directory/file.gz"infile = open(path)gz = Zlib::GzipReader.new(infile)gz.each_line do |line_content|          begin        JSON.parse(line_content)      rescue JSON::ParserError => e          p "json parsing exception" + " -- " + line_content.strip    rescue Exception => ex        puts "An error of type #{ex.class} happened, message is #{ex.message}"    end end

It gave me exception message for some lines in the file:

An error of type Encoding::InvalidByteSequenceError happened, message is "\xC3" on US-ASCII

Tried modifying my query according to this: https://community.hortonworks.com/articles/58548/processing-files-in-hive-using-native-non-utf8-cha.htmlModification was adding this line after creating external table:

 ALTER TABLE table_name SET SERDEPROPERTIES ('serialization.encoding'='SJIS');

It dint worked.P.S. : this is not empty files issue as discussed in some online forums, as I dont have blank files in hdfs.

What can i do about this, it seems like its some character encoding issue, but not able to get the solution.

In my angular application, I need to make GET call to a Tomcat server. This GET call requires query parameters which could contain special characters too like "+", "/", "/+"GET call is being made from angular controller using $window.open with target as "_blank"

Currently the redirection is getting failed without any encoding.

So, I added encoding in .js file before the GET call is being made by using encodeURIComponent.

Then I added decoding logic using URLDecode.decode in backend java code to decode query parameters.

But still it doesn't work.

It works only if I encode query parameters twice within the .js file using encodeURIComponent twice.

I am trying to find the root cause for double encoding but no luck yet. I would greatly appreciate if anyone could share any inputs.

I have a url that contains cyrillic symbols

I'm using the code below:

from selenium import webdriverfrom selenium.webdriver.firefox.firefox_binary import FirefoxBinaryimport timeimport urllib.requestfrom urllib.parse import quoteurl = 'http://username:password@isa-101.ut.uk:676/portal/d_table_view.php?tk=&y=2018&m=08&v='+ quote('cyrillic word')+'&b=[object NodeList]&g=coach&p='+quote('cyrillic word')print (url)driver = webdriver.Firefox(executable_path="C:/Users/Username/Desktop/screen/geckodriver.exe")  driver.set_window_size(1420,1720)driver.get(url)time.sleep(5)

in the console I get url, and in the browser it opens:1

but when i run my code , and insert the url below, it does not open:2

I have tried

url = url.encode('cp1251')url = url.decode('utf-8')

and

url = url.encode('utf-8')url = url.decode('utf-8')

But it did not help me, who knows how to fix it?

Is there a built-in way to URL encode a string in Excel VBA or do I need to hand roll this functionality?

Im am really new with this language and I need some help with this:

urldos = url + urllib.parse.urlencode({'placeid': linea}) + '&key=##########'

The output gives me a line like this:

https://maps.googleapis.com/maps/api/place/details/json?placeid=ChIJRTy9wHx82oURB9YQyTG4mSg%0A&key=#######'

And it seems that this little characters (%0A) are preventing my code to function in a correct way. So anyone has any idea?