FTP Unicode

InsightConnect Plugins
1

Hi I made a workflow that fetches list of attachments from MS 365, when the attachment is a PDF it will upload it on an FTP server locally that we setup. for some reason when the PDF filename is on a unicode the FTP Plugin fails to upload this is the error

Connect: Connecting…
rapid7/FTP:2.0.1. Step name: upload
‘latin-1’ codec can’t encode character ‘\uff0d’ in position 17: ordinal not in range(256)
Traceback (most recent call last):
File “/usr/local/site-packages/komand-1.0.1-py3.5.egg/komand/plugin.py”, line 307, in handle_step
output = self.start_step(input_message[‘body’], ‘action’, logger, log_stream, is_test, is_debug)
File “/usr/local/site-packages/komand-1.0.1-py3.5.egg/komand/plugin.py”, line 415, in start_step
output = func(params)
File “/usr/local/site-packages/ftp_rapid7_plugin-2.0.1-py3.5.egg/komand_ftp/actions/upload/action.py”, line 28, in run
self.connection.ftp_host.upload(tmp_file, remote_path)
File “/usr/local/site-packages/ftputil/host.py”, line 490, in upload
conditional=False, callback=callback)
File “/usr/local/site-packages/ftputil/file_transfer.py”, line 184, in copy_file
target_fobj = target_file.fobj()
File “/usr/local/site-packages/ftputil/file_transfer.py”, line 94, in fobj
return self._host.open(self.name, self.mode)
File “/usr/local/site-packages/ftputil/host.py”, line 242, in open
rest=rest)
File “/usr/local/site-packages/ftputil/file.py”, line 96, in _open
self._conn = self._session.transfercmd(command, rest)
File “/usr/local/site-packages/ftputil/file.py”, line 96, in _open
self._conn = self._session.transfercmd(command, rest)
File “/usr/local/lib-python/3/ftplib.py”, line 398, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File “/usr/local/lib-python/3/ftplib.py”, line 364, in ntransfercmd
resp = self.sendcmd(cmd)
File “/usr/local/lib-python/3/ftplib.py”, line 271, in sendcmd
self.putcmd(cmd)
File “/usr/local/lib-python/3/ftplib.py”, line 198, in putcmd
self.putline(line)
File “/usr/local/lib-python/3/ftplib.py”, line 193, in putline
self.sock.sendall(line.encode(self.encoding))
UnicodeEncodeError: ‘latin-1’ codec can’t encode character ‘\uff0d’ in position 17: ordinal not in range(256)

Please Help

2

Hi @xkiok_kan1 Thanks for letting us know, and we need to improve the error message for this action to be more helpful.

Two things of note,

  • The FTP plugin’s Upload action requires a base64 encoded version of the file. I wanted to check if you ran it through the base64 plugin before uploading (assuming it wasn’t encoded already)?
  • If the filename input is the issue you can use the Strings plugin to convert a unicode name to ASCII via the Set Encoding action Rapid7 Extensions

Let us know if these work for you

3

Hello its the filename that is having problem, the file contents is base64 encoded it came from the Microsoft 365 Mail plugin (icon_mail) , i’ll give the 2nd one a try but will this change the name of the file?

4

Hello, sorry but Using the String Encoding to ASCII does not help, setting Error Handling to the following gives unsatisfactory result:
replace - replaces the unsupported string to ???
ignore - removes the unsupported string
strict - raises an error.

We need the file_name to be preserved.