diff --git a/.gitignore b/.gitignore index 804eb70..358e9e5 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ -bower_components/* -node_modules/* \ No newline at end of file +_gitignore/ +bower_components/ +node_modules/ \ No newline at end of file diff --git a/LICENSE b/LICENSE index 5402f7c..0d04dcd 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2013 Matt +Copyright (c) 2014 Matthew Holt Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in diff --git a/README.md b/README.md index f25fd1f..1eaa710 100644 --- a/README.md +++ b/README.md @@ -3,30 +3,34 @@ Parse CSV with Javascript [![mholt on Gittip](http://img.shields.io/badge/tips-accepted-brightgreen.svg?style=flat)](https://www.gittip.com/mholt/) - Papa Parse (formerly the jQuery Parse Plugin) is a robust and powerful CSV (character-separated values) parser with these features: -- Parses delimited text strings without any fuss -- Attach to `` elements to load and parse files from disk -- Automatically detects delimiter (or specify a delimiter yourself) -- Supports streaming large inputs -- Utilize the header row, if present -- Gracefully handles malformed data -- Optional dynamic typing so that numeric data is parsed as numbers -- Descriptive and contextual errors +- Easy to use +- Parse CSV files directly (local or over the network) +- Stream large files (even via HTTP) +- Reverse parsing (converts JSON to CSV) +- Auto-detects the delimiter +- Worker threads to keep your web page responsive +- Header row support +- Can convert numbers and booleans to their types +- Graceful and robust error handling +- Minor jQuery integration to get files from `` elements + +All are optional (except for being easy to use). + -Demo +[Demo](http://papaparse.com/demo.html) ---- -Visit **[PapaParse.com](http://papaparse.com/#demo)** to give Papa a whirl! +Visit **[PapaParse.com/demo.html](http://papaparse.com/demo.html)** to try Papa! Get Started ----------- -Use [jquery.parse.min.js](https://github.com/mholt/jquery.parse/blob/master/jquery.parse.min.js) for production. +Use [papaparse.min.js](https://github.com/mholt/PapaParse/blob/master/papaparse.min.js) for production. For usage instructions, see the [homepage](http://papaparse.com) and, for more detail, the [documentation](http://papaparse.com/docs.html). @@ -35,18 +39,18 @@ For usage instructions, see the [homepage](http://papaparse.com) and, for more d Tests ----- -The Parser component is under test. Download this repository and open `tests.html` in your browser to run them. +Papa Parse is under test (especially its core Parser). Download this repository and open `tests/tests.html` in your browser to run them. Contributing ------------ -If you'd like to see a feature or bug fix, pull down the code and submit a pull request. But remember, if you're changing anything in the Parser function, a pull request, *with test*, is best. (All changes to the parser component should be validated with tests.) You may also open issues for discussion or join in on Twitter with [#PapaParse](https://twitter.com/search?q=%23PapaParse&src=typd&f=realtime) +To discuss a new feature or ask a question, open an issue. To fix a bug, submit a pull request to be credited with the [contributors](https://github.com/mholt/PapaParse/graphs/contributors)! Remember, a pull request, *with test*, is best. (Especially all changes to the Parser component should be validated with tests.) You may also discuss on Twitter with [#PapaParse](https://twitter.com/search?q=%23PapaParse&src=typd&f=realtime) or directly to me, [@mholt6](https://twitter.com/mholt6). Origins ------- -Papa Parse is the result of an experiment by [SmartyStreets](http://smartystreets.com) which matured into a fully-featured, independent jQuery plugin. Wanting to enhance the demo on their homepage, SmartyStreets looked into ways to simulate their list service. This involved processing at least part of a potentially large delimited text file. And what else? They wanted to do it without requiring a file upload (for simplicity and to alleviate privacy concerns). No suitable solutions were found, so they built their own. After finding it successful, the code was brought out into this jQuery plugin, now known as Papa Parse. +Papa Parse is the result of a successful experiment by [SmartyStreets](http://smartystreets.com) which matured into a fully-featured, independent Javascript library. diff --git a/bower.json b/bower.json index 4aebbb3..7207138 100644 --- a/bower.json +++ b/bower.json @@ -1,6 +1,6 @@ { "name": "Papa-Parse", - "main": "jquery.parse.js", + "main": "papaparse.js", "homepage": "http://papaparse.com", "authors": [ "Matthew Holt" @@ -20,7 +20,11 @@ "pipe", "file", "filereader", - "stream" + "stream", + "workers", + "ajax", + "threading", + "multi-threaded" ], "license": "MIT", "ignore": [ @@ -28,9 +32,7 @@ "node_modules", "bower_components", "test", - "tests" - ], - "dependencies": { - "jquery": ">=1.6.0" - } + "tests", + "player" + ] } diff --git a/jquery.parse.js b/jquery.parse.js deleted file mode 100644 index b55f768..0000000 --- a/jquery.parse.js +++ /dev/null @@ -1,677 +0,0 @@ -/* - Papa Parse - v2.1.4 - https://github.com/mholt/jquery.parse -*/ - -(function($) -{ - "use strict"; - - $.fn.parse = function(options) - { - var config = options.config || {}; - var queue = []; - - this.each(function(idx) - { - var supported = $(this).prop('tagName').toUpperCase() == "INPUT" - && $(this).attr('type') == "file" - && window.FileReader; - - if (!supported) - return true; // continue to next input element - - var instanceConfig = $.extend({}, config); // This copy is very important - - if (!this.files || this.files.length == 0) - { - error("NoFileError", undefined, this); - return true; // continue to next input element - } - - for (var i = 0; i < this.files.length; i++) - queue.push({ - file: this.files[i], - inputElem: this, - instanceConfig: instanceConfig - }); - - if (queue.length > 0) - parseFile(queue[0]); - }); - - return this; - - - function parseFile(f) - { - var completeFunc = complete, errorFunc; - - if (isFunction(options.error)) - errorFunc = function() { options.error(reader.error, f.file, f.inputElem); }; - if (isFunction(options.complete)) - completeFunc = function(results, file, inputElem, event) { options.complete(results, file, inputElem, event); complete(); }; - - if (isFunction(options.before)) - { - var returned = options.before(f.file, f.inputElem); - - if (typeof returned === 'object') - f.instanceConfig = $.extend(f.instanceConfig, returned); - else if (returned === "skip") - return complete(); // Proceeds to next file - else if (returned === false) - { - error("AbortError", f.file, f.inputElem); - return; // Aborts all queued files immediately - } - } - - if (f.instanceConfig.step) - { - var streamer = new Streamer(f.file, { - inputElem: f.inputElem, - config: $.extend({}, f.instanceConfig) // This copy is very important - }); - streamer.stream(completeFunc, errorFunc); - } - else - { - var reader = new FileReader(); - reader.onerror = errorFunc; - reader.onload = function(event) - { - var text = event.target.result; - var results = $.parse(text, f.instanceConfig); - completeFunc(results, f.file, f.inputElem, event); - }; - reader.readAsText(f.file, f.instanceConfig.encoding); - } - } - - function error(name, file, elem) - { - if (isFunction(options.error)) - options.error({name: name}, file, elem); - } - - function complete() - { - queue.splice(0, 1); - if (queue.length > 0) - parseFile(queue[0]); - } - }; - - $.parse = function(input, options) - { - var parser = new Parser(options); - return parser.parse(input); - }; - - function isFunction(func) { return typeof func === 'function'; } - - // Streamer is a wrapper over Parser to handle chunking the input file - function Streamer(file, settings) - { - if (!settings) - settings = {}; - - if (!settings.chunkSize) - settings.chunkSize = 1024 * 1024 * 5; // 5 MB - - if (settings.config.step) // it had better be there...! - { - var userStep = settings.config.step; - settings.config.step = function(data) { return userStep(data, file, settings.inputElem); }; - } - - var start = 0; - var aggregate = ""; - var partialLine = ""; - var parser = new Parser(settings.config); - var reader = new FileReader(); - - reader.onload = blobLoaded; - reader.onerror = blobError; - - this.stream = function(completeCallback, fileErrorCallback) - { - settings.onComplete = completeCallback; - settings.onFileError = fileErrorCallback; - nextChunk(); - }; - - function blobLoaded(event) - { - aggregate += partialLine + event.target.result; - partialLine = ""; - - if (start < file.size) - { - var lastLineEnd = aggregate.lastIndexOf("\n"); - - if (lastLineEnd < 0) - lastLineEnd = aggregate.lastIndexOf("\r"); - - if (lastLineEnd > -1) - { - partialLine = aggregate.substring(lastLineEnd + 1); // skip the line ending character - aggregate = aggregate.substring(0, lastLineEnd); - } - else - { - // For chunk sizes smaller than a line (a line could not fit in a single chunk) - // we simply build our aggregate by reading in the next chunk, until we find a newline - nextChunk(); - return; - } - } - - var results = parser.parse(aggregate); - aggregate = ""; - - if (start >= file.size) - return done(event); - else if (results.errors.abort) - return; - else - nextChunk(); - } - - function done(event) - { - if (typeof settings.onComplete === 'function') - settings.onComplete(undefined, file, settings.inputElem, event); - } - - function blobError() - { - if (typeof settings.onFileError === 'function') - settings.onFileError(reader.error, file, settings.inputElem); - } - - function nextChunk() - { - if (start < file.size) - { - var end = Math.min(start + settings.chunkSize, file.size); - - if (file.slice) { - reader.readAsText(file.slice(start, end), settings.config.encoding); - } - else if (file.webkitSlice) { - reader.readAsText(file.webkitSlice(start, end), settings.config.encoding); - } - - start += settings.chunkSize; - } - }; - } - - // Parser is the actual parsing component. - // It is under test and does not depend on jQuery. - // You could rip this entire function out of the plugin - // and use it independently (with attribution). - function Parser(config) - { - var self = this; - var _invocations = 0; - var _input = ""; - var _chunkOffset = 0; - var _abort = false; - var _config = {}; - var _state = freshState(); - var _defaultConfig = { - delimiter: "", - header: true, - dynamicTyping: true, - preview: 0, - commentChar: false - }; - var _regex = { - floats: /^\s*-?(\d*\.?\d+|\d+\.?\d*)(e[-+]?\d+)?\s*$/i, - empty: /^\s*$/ - }; - - config = validConfig(config); - _config = { - delimiter: config.delimiter, - header: config.header, - dynamicTyping: config.dynamicTyping, - preview: config.preview, - step: config.step, - commentChar: config.commentChar - }; - - this.parse = function(input) - { - if (typeof input !== 'string') - return returnable(); - - reset(input); - - if (!_config.delimiter && !guessDelimiter(input)) - { - addError("Delimiter", "UndetectableDelimiter", "Unable to auto-detect delimiting character; defaulted to comma", "config"); - _config.delimiter = ","; - } - - for (_state.i = 0; _state.i < _input.length; _state.i++) - { - if (_abort || (_config.preview > 0 && _state.lineNum > _config.preview)) - break; - - _state.ch = _input[_state.i]; - - if (_config.commentChar) { - // Check if line begins with a commentChar - if (_state.line == "" &&_state.ch == _config.commentChar) { - newRow(); - - // skip to next row - while (true) { - ++_state.i - if (_input[_state.i] == "\r" || _input[_state.i] == "\n") - break; - } - } - - _state.ch = _input[_state.i]; - - } - - _state.line += _state.ch; - - if (_state.ch == '"') - handleQuote(); - else if (_state.inQuotes) - inQuotes(); - else - notInQuotes(); - } - - if (_abort) - addError("Abort", "ParseAbort", "Parsing was aborted by the user's step function", "abort"); - else - { - endRow(); // End of input is also end of the last row - if (_state.inQuotes) - addError("Quotes", "MissingQuotes", "Unescaped or mismatched quotes"); - } - - return returnable(); - }; - - this.getOptions = function() - { - return { - delimiter: _config.delimiter, - header: _config.header, - dynamicTyping: _config.dynamicTyping, - preview: _config.preview, - step: _config.step - }; - }; - - function validConfig(config) - { - if (typeof config !== 'object') - config = {}; - - if (typeof config.delimiter !== 'string' - || config.delimiter.length != 1) - config.delimiter = _defaultConfig.delimiter; - - if (config.delimiter == '"' || config.delimiter == "\n") - config.delimiter = _defaultConfig.delimiter; - - if (typeof config.header !== 'boolean') - config.header = _defaultConfig.header; - - if (typeof config.dynamicTyping !== 'boolean') - config.dynamicTyping = _defaultConfig.dynamicTyping; - - if (typeof config.preview !== 'number') - config.preview = _defaultConfig.preview; - - if (typeof config.step !== 'function') - config.step = _defaultConfig.step; - - if (config.commentChar === true) - config.commentChar = '#'; - - if (typeof config.commentChar !== 'string' && config.commentChar !== false) - config.commentChar = false; - - return config; - } - - function guessDelimiter(input) - { - var recordSep = String.fromCharCode(30); - var unitSep = String.fromCharCode(31); - var delimiters = [",", "\t", "|", ";", recordSep, unitSep]; - var bestDelim, bestDelta, fieldCountPrevRow; - - for (var i = 0; i < delimiters.length; i++) - { - var delim = delimiters[i]; - var delta = 0, avgFieldCount = 0; - var fieldCountPrevRow = undefined; - - var preview = new Parser({ - delimiter: delim, - header: false, - dynamicTyping: false, - preview: 10 - }).parse(input); - - for (var j = 0; j < preview.results.length; j++) - { - var fieldCount = preview.results[j].length; - avgFieldCount += fieldCount; - - if (typeof fieldCountPrevRow === 'undefined') - { - fieldCountPrevRow = fieldCount; - continue; - } - else if (fieldCount > 1) - { - delta += Math.abs(fieldCount - fieldCountPrevRow); - fieldCountPrevRow = fieldCount; - } - } - - avgFieldCount /= preview.results.length; - - if ((typeof bestDelta === 'undefined' || delta < bestDelta) - && avgFieldCount > 1.99) - { - bestDelta = delta; - bestDelim = delim; - } - } - - _config.delimiter = bestDelim; - - return !!bestDelim; - } - - function handleQuote() - { - var delimBefore = (_state.i > 0 && isBoundary(_state.i-1)) - || _state.i == 0; - var delimAfter = (_state.i < _input.length - 1 && isBoundary(_state.i+1)) - || _state.i == _input.length - 1; - var escaped = _state.i < _input.length - 1 - && _input[_state.i+1] == '"'; - - if (_state.inQuotes && escaped) - { - _state.fieldVal += '"'; - _state.i++; - } - else if (delimBefore || delimAfter) - _state.inQuotes = !_state.inQuotes; - else - addError("Quotes", "UnexpectedQuotes", "Unexpected quotes"); - } - - function inQuotes() - { - appendCharToField(); - } - - function appendCharToField() - { - _state.fieldVal += _state.ch; - } - - function notInQuotes() - { - if (_state.ch == _config.delimiter) - saveValue(); - else if ((_state.ch == "\r" && _state.i < _input.length - 1 - && _input[_state.i+1] == "\n") - || (_state.ch == "\n" && _state.i < _input.length - 1 - && _input[_state.i+1] == "\r")) - { - newRow(); - _state.i++; - } - else if (_state.ch == "\r" || _state.ch == "\n") - newRow(); - else - appendCharToField(); - } - - function isBoundary(i) - { - return _input[i] == _config.delimiter - || _input[i] == "\n" - || _input[i] == "\r"; - } - - function saveValue() - { - if (_config.header) - { - if (_state.lineNum == 1 && _invocations == 1) - _state.parsed.fields.push(_state.fieldVal); - else - { - var currentRow = _state.parsed.rows[_state.parsed.rows.length - 1]; - var fieldName = _state.parsed.fields[_state.field]; - if (fieldName) - { - if (_config.dynamicTyping) - _state.fieldVal = tryParseFloat(_state.fieldVal); - currentRow[fieldName] = _state.fieldVal; - } - else - { - if (typeof currentRow.__parsed_extra === 'undefined') - currentRow.__parsed_extra = []; - currentRow.__parsed_extra.push(_state.fieldVal); - } - } - } - else - { - if (_config.dynamicTyping) - _state.fieldVal = tryParseFloat(_state.fieldVal); - _state.parsed[_state.parsed.length - 1].push(_state.fieldVal); - } - - _state.fieldVal = ""; - _state.field ++; - } - - function newRow() - { - endRow(); - - if (streaming()) - { - _state.errors = {}; - _state.errors.length = 0; - } - - if (_config.header) - { - if (_state.lineNum > 0) - { - if (streaming()) - _state.parsed.rows = [ {} ]; - else - _state.parsed.rows.push({}); - } - } - else - { - if (streaming()) - _state.parsed = [ [] ]; - else if (!_config.header) - _state.parsed.push([]); - } - - _state.lineNum++; - _state.line = ""; - _state.field = 0; - } - - function endRow() - { - if (_abort) - return; - - saveValue(); - - var emptyLine = trimEmptyLine(); - - if (!emptyLine && _config.header) - inspectFieldCount(); - - if (streaming() && (!_config.header || - (_config.header && _state.parsed.rows.length > 0))) - { - var keepGoing = _config.step(returnable()); - if (keepGoing === false) - _abort = true; - } - } - - function streaming() - { - return typeof _config.step === 'function'; - } - - function tryParseFloat(num) - { - var isNumber = _regex.floats.test(num); - return isNumber ? parseFloat(num) : num; - } - - function trimEmptyLine() - { - if (_regex.empty.test(_state.line)) - { - if (_config.header) - { - if (_state.lineNum == 1) - { - _state.parsed.fields = []; - _state.lineNum--; - } - else - _state.parsed.rows.splice(_state.parsed.rows.length - 1, 1); - } - else - _state.parsed.splice(_state.parsed.length - 1, 1); - - return true; - } - return false; - } - - function inspectFieldCount() - { - if (!_config.header) - return true; - - if (_state.parsed.rows.length == 0) - return true; - - var expected = _state.parsed.fields.length; - - // Actual field count tabulated manually because IE<9 doesn't support Object.keys - var actual = 0; - var lastRow = _state.parsed.rows[_state.parsed.rows.length - 1]; - for (var prop in lastRow) - if (lastRow.hasOwnProperty(prop)) - actual++; - - if (actual < expected) - return addError("FieldMismatch", "TooFewFields", "Too few fields: expected " + expected + " fields but parsed " + actual); - else if (actual > expected) - return addError("FieldMismatch", "TooManyFields", "Too many fields: expected " + expected + " fields but parsed " + actual); - return true; - } - - function addError(type, code, msg, errKey) - { - var row = _config.header - ? (_state.parsed.rows.length ? _state.parsed.rows.length - 1 : undefined) - : _state.parsed.length - 1; - var key = errKey || row; - - if (typeof _state.errors[key] === 'undefined') - _state.errors[key] = []; - - _state.errors[key].push({ - type: type, - code: code, - message: msg, - line: _state.lineNum, - row: row, - index: _state.i + _chunkOffset - }); - - _state.errors.length ++; - - return false; - } - - function returnable() - { - return { - results: _state.parsed, - errors: _state.errors, - meta: { - delimiter: _config.delimiter - } - }; - } - - function reset(input) - { - _invocations++; - if (_invocations > 1 && streaming()) - _chunkOffset += input.length; - _state = freshState(); - _input = input; - } - - function freshState() - { - // If streaming, and thus parsing the input in chunks, this - // is careful to preserve what we've already got, when necessary. - var parsed; - if (_config.header) - { - parsed = { - fields: streaming() ? _state.parsed.fields || [] : [], - rows: streaming() && _invocations > 1 ? [ {} ] : [] - }; - } - else - parsed = [ [] ]; - - return { - i: 0, - lineNum: streaming() ? _state.lineNum : 1, - field: 0, - fieldVal: "", - line: "", - ch: "", - inQuotes: false, - parsed: parsed, - errors: { length: 0 } - }; - } - } - -})(jQuery); diff --git a/jquery.parse.min.js b/jquery.parse.min.js deleted file mode 100644 index 67a6d92..0000000 --- a/jquery.parse.min.js +++ /dev/null @@ -1,6 +0,0 @@ -/* - Papa Parse - v2.1.4 - https://github.com/mholt/jquery.parse -*/ -;(function(e){"use strict";function t(e){return typeof e==="function"}function n(e,t){function f(t){s+=o+t.target.result;o="";if(i-1){o=s.substring(n+1);s=s.substring(0,n)}else{h();return}}var r=u.parse(s);s="";if(i>=e.size)return l(t);else if(r.errors.abort)return;else h()}function l(n){if(typeof t.onComplete==="function")t.onComplete(undefined,e,t.inputElem,n)}function c(){if(typeof t.onFileError==="function")t.onFileError(a.error,e,t.inputElem)}function h(){if(i1){c+=Math.abs(v-a);a=v}}h/=p.results.length;if((typeof o==="undefined"||c1.99){o=c;s=l}}u.delimiter=s;return!!s}function p(){var e=a.i>0&&g(a.i-1)||a.i==0;var t=a.i0){if(E())a.parsed.rows=[{}];else a.parsed.rows.push({})}}else{if(E())a.parsed=[[]];else if(!u.header)a.parsed.push([])}a.lineNum++;a.line="";a.field=0}function w(){if(o)return;y();var e=x();if(!e&&u.header)T();if(E()&&(!u.header||u.header&&a.parsed.rows.length>0)){var t=u.step(C());if(t===false)o=true}}function E(){return typeof u.step==="function"}function S(e){var t=l.floats.test(e);return t?parseFloat(e):e}function x(){if(l.empty.test(a.line)){if(u.header){if(a.lineNum==1){a.parsed.fields=[];a.lineNum--}else a.parsed.rows.splice(a.parsed.rows.length-1,1)}else a.parsed.splice(a.parsed.length-1,1);return true}return false}function T(){if(!u.header)return true;if(a.parsed.rows.length==0)return true;var e=a.parsed.fields.length;var t=0;var n=a.parsed.rows[a.parsed.rows.length-1];for(var r in n)if(n.hasOwnProperty(r))t++;if(te)return N("FieldMismatch","TooManyFields","Too many fields: expected "+e+" fields but parsed "+t);return true}function N(e,t,n,r){var i=u.header?a.parsed.rows.length?a.parsed.rows.length-1:undefined:a.parsed.length-1;var o=r||i;if(typeof a.errors[o]==="undefined")a.errors[o]=[];a.errors[o].push({type:e,code:t,message:n,line:a.lineNum,row:i,index:a.i+s});a.errors.length++;return false}function C(){return{results:a.parsed,errors:a.errors,meta:{delimiter:u.delimiter}}}function k(e){n++;if(n>1&&E())s+=e.length;a=L();i=e}function L(){var e;if(u.header){e={fields:E()?a.parsed.fields||[]:[],rows:E()&&n>1?[{}]:[]}}else e=[[]];return{i:0,lineNum:E()?a.lineNum:1,field:0,fieldVal:"",line:"",ch:"",inQuotes:false,parsed:e,errors:{length:0}}}var t=this;var n=0;var i="";var s=0;var o=false;var u={};var a=L();var f={delimiter:"",header:true,dynamicTyping:true,preview:0};var l={floats:/^\s*-?(\d*\.?\d+|\d+\.?\d*)(e[-+]?\d+)?\s*$/i,empty:/^\s*$/};e=c(e);u={delimiter:e.delimiter,header:e.header,dynamicTyping:e.dynamicTyping,preview:e.preview,step:e.step};this.parse=function(e){if(typeof e!=="string")return C();k(e);if(!u.delimiter&&!h(e)){N("Delimiter","UndetectableDelimiter","Unable to auto-detect delimiting character; defaulted to comma","config");u.delimiter=","}for(a.i=0;a.i0&&a.lineNum>u.preview)break;a.ch=i[a.i];a.line+=a.ch;if(a.ch=='"')p();else if(a.inQuotes)d();else m()}if(o)N("Abort","ParseAbort","Parsing was aborted by the user's step function","abort");else{w();if(a.inQuotes)N("Quotes","MissingQuotes","Unescaped or mismatched quotes")}return C()};this.getOptions=function(){return{delimiter:u.delimiter,header:u.header,dynamicTyping:u.dynamicTyping,preview:u.preview,step:u.step}}}e.fn.parse=function(r){function o(i){var s=a,o;if(t(r.error))o=function(){r.error(c.error,i.file,i.inputElem)};if(t(r.complete))s=function(e,t,n,i){r.complete(e,t,n,i);a()};if(t(r.before)){var f=r.before(i.file,i.inputElem);if(typeof f==="object")i.instanceConfig=e.extend(i.instanceConfig,f);else if(f==="skip")return a();else if(f===false){u("AbortError",i.file,i.inputElem);return}}if(i.instanceConfig.step){var l=new n(i.file,{inputElem:i.inputElem,config:e.extend({},i.instanceConfig)});l.stream(s,o)}else{var c=new FileReader;c.onerror=o;c.onload=function(t){var n=t.target.result;var r=e.parse(n,i.instanceConfig);s(r,i.file,i.inputElem,t)};c.readAsText(i.file,i.instanceConfig.encoding)}}function u(e,n,i){if(t(r.error))r.error({name:e},n,i)}function a(){s.splice(0,1);if(s.length>0)o(s[0])}var i=r.config||{};var s=[];this.each(function(t){var n=e(this).prop("tagName").toUpperCase()=="INPUT"&&e(this).attr("type")=="file"&&window.FileReader;if(!n)return true;var r=e.extend({},i);if(!this.files||this.files.length==0){u("NoFileError",undefined,this);return true}for(var a=0;a0)o(s[0])});return this};e.parse=function(e,t){var n=new r(t);return n.parse(e)}})(jQuery); \ No newline at end of file diff --git a/papaparse.js b/papaparse.js new file mode 100644 index 0000000..1ee7b28 --- /dev/null +++ b/papaparse.js @@ -0,0 +1,1261 @@ +/* + Papa Parse + v3.0.0 + https://github.com/mholt/PapaParse +*/ +(function(global) +{ + "use strict"; + + var IS_WORKER = !global.document, SCRIPT_PATH; + var workers = {}, workerIdCounter = 0; + + // A configuration object from which to draw default settings + var DEFAULTS = { + delimiter: "", // empty: auto-detect + header: false, + dynamicTyping: false, + preview: 0, + step: undefined, + encoding: "", // browser should default to "UTF-8" + worker: false, + comments: false, + complete: undefined, + download: false + }; + + global.Papa = {}; + + global.Papa.parse = CsvToJson; + global.Papa.unparse = JsonToCsv; + + global.Papa.RECORD_SEP = String.fromCharCode(30); + global.Papa.UNIT_SEP = String.fromCharCode(31); + global.Papa.BYTE_ORDER_MARK = "\ufeff"; + global.Papa.BAD_DELIMITERS = ["\r", "\n", "\"", global.Papa.BYTE_ORDER_MARK]; + global.Papa.WORKERS_SUPPORTED = !!global.Worker; + + // Exposed for testing and development only + global.Papa.Parser = Parser; + global.Papa.ParserHandle = ParserHandle; + global.Papa.NetworkStreamer = NetworkStreamer; + global.Papa.FileStreamer = FileStreamer; + + if (global.jQuery) + { + var $ = global.jQuery; + $.fn.parse = function(options) + { + var config = options.config || {}; + var queue = []; + + this.each(function(idx) + { + var supported = $(this).prop('tagName').toUpperCase() == "INPUT" + && $(this).attr('type').toLowerCase() == "file" + && global.FileReader; + + if (!supported || !this.files || this.files.length == 0) + return true; // continue to next input element + + for (var i = 0; i < this.files.length; i++) + { + queue.push({ + file: this.files[i], + inputElem: this, + instanceConfig: $.extend({}, config) + }); + } + }); + + parseNextFile(); // begin parsing + return this; // maintains chainability + + + function parseNextFile() + { + if (queue.length == 0) + { + if (isFunction(options.complete)) + options.complete(); + return; + } + + var f = queue[0]; + + if (isFunction(options.before)) + { + var returned = options.before(f.file, f.inputElem); + + if (typeof returned === 'object') + { + if (returned.action == "abort") + { + error("AbortError", f.file, f.inputElem, returned.reason); + return; // Aborts all queued files immediately + } + else if (returned.action == "skip") + { + fileComplete(); // parse the next file in the queue, if any + return; + } + else if (typeof returned.config === 'object') + f.instanceConfig = $.extend(f.instanceConfig, returned.config); + } + else if (returned == "skip") + { + fileComplete(); // parse the next file in the queue, if any + return; + } + } + + // Wrap up the user's complete callback, if any, so that ours also gets executed + var userCompleteFunc = f.instanceConfig.complete; + f.instanceConfig.complete = function(results) + { + if (isFunction(userCompleteFunc)) + userCompleteFunc(results, f.file, f.inputElem); + fileComplete(); + }; + + Papa.parse(f.file, f.instanceConfig); + } + + function error(name, file, elem, reason) + { + if (isFunction(options.error)) + options.error({name: name}, file, elem, reason); + } + + function fileComplete() + { + queue.splice(0, 1); + parseNextFile(); + } + } + } + + + if (IS_WORKER) + global.onmessage = workerThreadReceivedMessage; + else if (Papa.WORKERS_SUPPORTED) + SCRIPT_PATH = getScriptPath(); + + + + + function CsvToJson(_input, _config) + { + var config = IS_WORKER ? _config : copyAndValidateConfig(_config); + var useWorker = config.worker && Papa.WORKERS_SUPPORTED; + + if (useWorker) + { + var w = newWorker(); + + w.userStep = config.step; + w.userComplete = config.complete; + w.userError = config.error; + + config.step = isFunction(config.step); + config.complete = isFunction(config.complete); + config.error = isFunction(config.error); + delete config.worker; // prevent infinite loop + + w.postMessage({ + input: _input, + config: config, + workerId: w.id + }); + } + else + { + if (typeof _input === 'string') + { + if (config.download) + { + var streamer = new NetworkStreamer(config); + streamer.stream(_input); + } + else + { + var ph = new ParserHandle(config); + var results = ph.parse(_input); + if (isFunction(config.complete)) + config.complete(results); + return results; + } + } + else if (_input instanceof File) + { + if (config.step) + { + var streamer = new FileStreamer(config); + streamer.stream(_input); + } + else + { + var ph = new ParserHandle(config); + + if (IS_WORKER) + { + var reader = new FileReaderSync(); + var input = reader.readAsText(_input, config.encoding); + return ph.parse(input); + } + else + { + reader = new FileReader(); + reader.onload = function(event) + { + var ph = new ParserHandle(config); + var results = ph.parse(event.target.result); + if (isFunction(config.complete)) + config.complete(results); + }; + reader.readAsText(_input, config.encoding); + } + } + } + } + } + + + + + + + function JsonToCsv(_input, _config) + { + var _output = ""; + var _fields = []; + + // Default configuration + var _quotes = false; // whether to surround every datum with quotes + var _delimiter = ","; // delimiting character + var _newline = "\r\n"; // newline character(s) + + unpackConfig(); + + if (typeof _input === 'string') + _input = JSON.parse(_input); + + if (_input instanceof Array) + { + if (!_input.length || _input[0] instanceof Array) + return serialize(null, _input); + else if (typeof _input[0] === 'object') + return serialize(objectKeys(_input[0]), _input); + } + else if (typeof _input === 'object') + { + if (typeof _input.data === 'string') + _input.data = JSON.parse(_input.data); + + if (_input.data instanceof Array) + { + if (!_input.fields) + _input.fields = _input.data[0] instanceof Array + ? _input.fields + : objectKeys(_input.data[0]); + + if (!(_input.data[0] instanceof Array) && typeof _input.data[0] !== 'object') + _input.data = [_input.data]; // handles input like [1,2,3] or ["asdf"] + + return serialize(_input.fields, _input.data); + } + } + + // Default (any valid paths should return before this) + throw "exception: Unable to serialize unrecognized input"; + + + function unpackConfig() + { + if (typeof _config !== 'object') + return; + + if (typeof _config.delimiter === 'string' + && _config.delimiter.length == 1 + && global.Papa.BAD_DELIMITERS.indexOf(_config.delimiter) == -1) + { + _delimiter = _config.delimiter; + } + + if (typeof _config.quotes === 'boolean') + _quotes = _config.quotes; + + if (typeof _config.newline === 'string') + _newline = _config.newline; + } + + + // Turns an object's keys into an array + function objectKeys(obj) + { + if (typeof obj !== 'object') + return []; + var keys = []; + for (var key in obj) + keys.push(key); + return keys; + } + + // The double for loop that iterates the data and writes out a CSV string including header row + function serialize(fields, data) + { + var csv = ""; + + if (typeof fields === 'string') + fields = JSON.parse(fields); + if (typeof data === 'string') + data = JSON.parse(data); + + var hasHeader = fields instanceof Array && fields.length > 0; + var dataKeyedByField = !(data[0] instanceof Array); + + // If there a header row, write it first + if (hasHeader) + { + for (var i = 0; i < fields.length; i++) + { + if (i > 0) + csv += _delimiter; + csv += safe(fields[i]); + } + if (data.length > 0) + csv += _newline; + } + + // Then write out the data + for (var row = 0; row < data.length; row++) + { + var maxCol = hasHeader ? fields.length : data[row].length; + + for (var col = 0; col < maxCol; col++) + { + if (col > 0) + csv += _delimiter; + var colIdx = hasHeader && dataKeyedByField ? fields[col] : col; + csv += safe(data[row][colIdx]); + } + + if (row < data.length - 1) + csv += _newline; + } + + return csv; + } + + // Encloses a value around quotes if needed (makes a value safe for CSV insertion) + function safe(str) + { + if (typeof str === "undefined") + return ""; + + str = str.toString().replace(/"/g, '""'); + + var needsQuotes = _quotes + || hasAny(str, global.Papa.BAD_DELIMITERS) + || str.indexOf(_delimiter) > -1 + || str.charAt(0) == ' ' + || str.charAt(str.length - 1) == ' '; + + return needsQuotes ? '"' + str + '"' : str; + } + + function hasAny(str, substrings) + { + for (var i = 0; i < substrings.length; i++) + if (str.indexOf(substrings[i]) > -1) + return true; + return false; + } + } + + + + // NOTE/TODO: Many of the functions of NetworkStreamer and FileStreamer are the same. Consolidate? + function NetworkStreamer(config) + { + config = config || {}; + if (!config.chunkSize) + config.chunkSize = 1024 * 1024 * 5; // 5 MB + + var start = 0, fileSize = 0; + var aggregate = ""; + var partialLine = ""; + var xhr, nextChunk; + var handle = new ParserHandle(copy(config)); + + this.stream = function(url) + { + if (IS_WORKER) + { + nextChunk = function() + { + readChunk(); + chunkLoaded(); + }; + } + else + { + nextChunk = function() + { + readChunk(); + }; + } + + nextChunk(); // Starts streaming + + + function readChunk() + { + xhr = new XMLHttpRequest(); + if (!IS_WORKER) + { + xhr.onload = chunkLoaded; + xhr.onerror = chunkError; + } + xhr.open("GET", url, !IS_WORKER); + if (config.step) + { + var end = start + config.chunkSize - 1; // minus one because byte range is inclusive + if (fileSize && end > fileSize) // Hack around a Chrome bug: http://stackoverflow.com/q/24745095/1048862 + end = fileSize; + xhr.setRequestHeader("Range", "bytes="+start+"-"+end); + } + xhr.send(); + if (IS_WORKER && xhr.status == 0) + chunkError(); + else + start += config.chunkSize; + } + + function chunkLoaded() + { + if (xhr.readyState != 4) + return; + + if (xhr.status < 200 || xhr.status >= 400) + { + chunkError(); + return; + } + + // Rejoin the line we likely just split in two by chunking the file + aggregate += partialLine + xhr.responseText; + partialLine = ""; + + var finishedWithEntireFile = !config.step || start > getFileSize(xhr); + + if (!finishedWithEntireFile) + { + var lastLineEnd = aggregate.lastIndexOf("\n"); + + if (lastLineEnd < 0) + lastLineEnd = aggregate.lastIndexOf("\r"); + + if (lastLineEnd > -1) + { + partialLine = aggregate.substring(lastLineEnd + 1); // skip the line ending character + aggregate = aggregate.substring(0, lastLineEnd); + } + else + { + // For chunk sizes smaller than a line (a line could not fit in a single chunk) + // we simply build our aggregate by reading in the next chunk, until we find a newline + nextChunk(); + return; + } + } + + var results = handle.parse(aggregate); + aggregate = ""; + + if (IS_WORKER) + { + global.postMessage({ + results: results, + workerId: Papa.WORKER_ID, + finished: finishedWithEntireFile + }); + } + + if (finishedWithEntireFile && isFunction(config.complete)) + config.complete(results); + else if (results && results.meta.aborted && isFunction(config.complete)) + config.complete(results); + else if (!finishedWithEntireFile) + nextChunk(); + } + + function chunkError() + { + if (isFunction(config.error)) + config.error(xhr.statusText); + else if (IS_WORKER && config.error) + { + global.postMessage({ + workerId: Papa.WORKER_ID, + error: xhr.statusText, + finished: false + }); + } + } + + function getFileSize(xhr) + { + var contentRange = xhr.getResponseHeader("Content-Range"); + return parseInt(contentRange.substr(contentRange.lastIndexOf("/") + 1)); + } + }; + } + + + + + + + + + + function FileStreamer(config) + { + config = config || {}; + if (!config.chunkSize) + config.chunkSize = 1024 * 1024 * 10; // 10 MB + + var start = 0; + var aggregate = ""; + var partialLine = ""; + var reader, nextChunk, slice; + var handle = new ParserHandle(copy(config)); + + this.stream = function(file) + { + var slice = file.slice || file.webkitSlice || file.mozSlice; + + reader = new FileReader(); // Better than FileReaderSync (even in worker). See: http://stackoverflow.com/q/24708649/1048862 + reader.onload = chunkLoaded; + reader.onerror = chunkError; + + nextChunk(); // Starts streaming + + function nextChunk() + { + if (start < file.size) + readChunk(); + } + + function readChunk() + { + var end = Math.min(start + config.chunkSize, file.size); + var txt = reader.readAsText(slice.call(file, start, end), config.encoding); + start += config.chunkSize; + return txt; + } + + function chunkLoaded(event) + { + // Rejoin the line we likely just split in two by chunking the file + aggregate += partialLine + event.target.result; + partialLine = ""; + + var finishedWithEntireFile = start >= file.size; + + if (!finishedWithEntireFile) + { + var lastLineEnd = aggregate.lastIndexOf("\n"); + + if (lastLineEnd < 0) + lastLineEnd = aggregate.lastIndexOf("\r"); + + if (lastLineEnd > -1) + { + partialLine = aggregate.substring(lastLineEnd + 1); // skip the line ending character + aggregate = aggregate.substring(0, lastLineEnd); + } + else + { + // For chunk sizes smaller than a line (a line could not fit in a single chunk) + // we simply build our aggregate by reading in the next chunk, until we find a newline + nextChunk(); + return; + } + } + + var results = handle.parse(aggregate); + aggregate = ""; + + if (IS_WORKER) + { + global.postMessage({ + results: results, + workerId: Papa.WORKER_ID, + finished: finishedWithEntireFile + }); + } + + if (finishedWithEntireFile && isFunction(config.complete)) + config.complete(undefined, file); + else if (results.meta.aborted && isFunction(config.complete)) + config.complete(results, file); + else if (!results.meta.paused) + nextChunk(); + } + + function chunkError() + { + if (isFunction(config.error)) + config.error(reader.error, file); + else if (IS_WORKER && config.error) + { + global.postMessage({ + workerId: Papa.WORKER_ID, + error: reader.error, + file: file, + finished: false + }); + } + } + }; + } + + + + + + // Use one ParserHandle per entire CSV file or string + function ParserHandle(_config) + { + // One goal is to minimize the use of regular expressions... + var FLOAT = /^\s*-?(\d*\.?\d+|\d+\.?\d*)(e[-+]?\d+)?\s*$/i; + + var _delimiterError; // Temporary state between delimiter detection and processing results + var _fields = []; // Fields are from the header row of the input, if there is one + var _results = { // The last results returned from the parser + data: [], + errors: [], + meta: {} + }; + _config = copy(_config); + + this.parse = function(input) + { + _delimiterError = false; + if (!_config.delimiter) + { + var delimGuess = guessDelimiter(input); + if (delimGuess.successful) + _config.delimiter = delimGuess.bestDelimiter; + else + { + _delimiterError = true; // add error after parsing (otherwise it would be overwritten) + _config.delimiter = ","; + } + _results.meta.delimiter = _config.delimiter; + } + + if (isFunction(_config.step)) + { + var userStep = _config.step; + _config.step = function(results, parser) + { + _results = results; + if (needsHeaderRow()) + processResults(); + else + userStep(processResults(), parser); + }; + } + + _results = new Parser(_config).parse(input); + return processResults(); + }; + + function processResults() + { + if (_results && _delimiterError) + { + addError("Delimiter", "UndetectableDelimiter", "Unable to auto-detect delimiting character; defaulted to comma"); + _delimiterError = false; + } + + if (needsHeaderRow()) + fillHeaderFields(); + + return applyHeaderAndDynamicTyping(); + } + + function needsHeaderRow() + { + return _config.header && _fields.length == 0; + } + + function fillHeaderFields() + { + if (!_results) + return; + for (var i = 0; needsHeaderRow() && i < _results.data.length; i++) + for (var j = 0; j < _results.data[i].length; j++) + _fields.push(_results.data[i][j]); + _results.data.splice(0, 1); + } + + function applyHeaderAndDynamicTyping() + { + if (!_results || (!_config.header && !_config.dynamicTyping)) + return _results; + + for (var i = 0; i < _results.data.length; i++) + { + var row = {}; + for (var j = 0; j < _results.data[i].length; j++) + { + if (_config.dynamicTyping) + { + var value = _results.data[i][j]; + if (value == "true") + _results.data[i][j] = true; + else if (value == "false") + _results.data[i][j] = false; + else + _results.data[i][j] = tryParseFloat(value); + } + + if (_config.header) + { + if (j >= _fields.length) + { + if (!row["__parsed_extra"]) + row["__parsed_extra"] = []; + row["__parsed_extra"].push(_results.data[i][j]); + } + row[_fields[j]] = _results.data[i][j]; + } + } + + if (_config.header) + { + _results.data[i] = row; + if (j > _fields.length) + addError("FieldMismatch", "TooManyFields", "Too many fields: expected " + _fields.length + " fields but parsed " + j, i); + else if (j < _fields.length) + addError("FieldMismatch", "TooFewFields", "Too few fields: expected " + _fields.length + " fields but parsed " + j, i); + } + } + + if (_config.header && _results.meta); + _results.meta.fields = _fields; + + return _results; + } + + function guessDelimiter(input) + { + var delimChoices = [",", "\t", "|", ";", Papa.RECORD_SEP, Papa.UNIT_SEP]; + var bestDelim, bestDelta, fieldCountPrevRow; + + for (var i = 0; i < delimChoices.length; i++) + { + var delim = delimChoices[i]; + var delta = 0, avgFieldCount = 0; + fieldCountPrevRow = undefined; + + var preview = new Parser({ + delimiter: delim, + preview: 10 + }).parse(input); + + for (var j = 0; j < preview.data.length; j++) + { + var fieldCount = preview.data[j].length; + avgFieldCount += fieldCount; + + if (typeof fieldCountPrevRow === 'undefined') + { + fieldCountPrevRow = fieldCount; + continue; + } + else if (fieldCount > 1) + { + delta += Math.abs(fieldCount - fieldCountPrevRow); + fieldCountPrevRow = fieldCount; + } + } + + avgFieldCount /= preview.data.length; + + if ((typeof bestDelta === 'undefined' || delta < bestDelta) + && avgFieldCount > 1.99) + { + bestDelta = delta; + bestDelim = delim; + } + } + + _config.delimiter = bestDelim; + + return { + successful: !!bestDelim, + bestDelimiter: bestDelim + } + } + + function tryParseFloat(val) + { + var isNumber = FLOAT.test(val); + return isNumber ? parseFloat(val) : val; + } + + function addError(type, code, msg, row) + { + _results.errors.push({ + type: type, + code: code, + message: msg, + row: row + }); + } + } + + + + + + + + + + + + function Parser(config) + { + var self = this; + var EMPTY = /^\s*$/; + + var _input; // The input text being parsed + var _delimiter; // The delimiting character + var _comments; // Comment character (default '#') or boolean + var _step; // The step (streaming) function + var _callback; // The callback to invoke when finished + var _preview; // Maximum number of lines (not rows) to parse + var _ch; // Current character + var _i; // Current character's positional index + var _inQuotes; // Whether in quotes or not + var _lineNum; // Current line number (1-based indexing) + var _data; // Parsed data (results) + var _errors; // Parse errors + var _rowIdx; // Current row index within results (0-based) + var _colIdx; // Current col index within result row (0-based) + var _aborted = false; // Abort flag + var _paused = false; // Pause flag + + // Unpack the config object + config = config || {}; + _delimiter = config.delimiter; + _comments = config.comments; + _step = config.step; + _preview = config.preview; + + // Delimiter integrity check + if (typeof _delimiter !== 'string' + || _delimiter.length != 1 + || Papa.BAD_DELIMITERS.indexOf(_delimiter) > -1) + _delimiter = ","; + + // Comment character integrity check + if (_comments === true) + _comments = "#"; + else if (typeof _comments !== 'string' + || _comments.length != 1 + || Papa.BAD_DELIMITERS.indexOf(_comments) > -1 + || _comments == _delimiter) + _comments = false; + + + this.parse = function(input) + { + if (typeof input !== 'string') + throw "Input must be a string"; + reset(input); + return parserLoop(); + }; +/* + // TODO: Pause and resume just doesn't work well. + // I suspect this may need to be implemented at a higher-level + // scope than just this core Parser. + this.pause = function() + { + _paused = true; + }; + + this.resume = function() + { + _paused = false; + if (_i < _input.length) + return parserLoop(); + }; +*/ + this.abort = function() + { + _aborted = true; + }; + + function parserLoop() + { + while (_i < _input.length) + { + if (_aborted) break; + if (_preview > 0 && _rowIdx >= _preview) break; + if (_paused) return finishParsing(); + + if (_ch == '"') + parseQuotes(); + else if (_inQuotes) + parseInQuotes(); + else + parseNotInQuotes(); + + nextChar(); + } + + return finishParsing(); + } + + function nextChar() + { + _i++; + _ch = _input[_i]; + } + + function finishParsing() + { + if (_aborted) + addError("Abort", "ParseAbort", "Parsing was aborted by the user's step function"); + if (_inQuotes) + addError("Quotes", "MissingQuotes", "Unescaped or mismatched quotes"); + endRow(); // End of input is also end of the last row + if (!isFunction(_step)) + return returnable(); + } + + function parseQuotes() + { + if (quotesOnBoundary() && !quotesEscaped()) + _inQuotes = !_inQuotes; + else + { + saveChar(); + if (_inQuotes && quotesEscaped()) + _i++ + else + addError("Quotes", "UnexpectedQuotes", "Unexpected quotes"); + } + } + + function parseInQuotes() + { + saveChar(); + } + + function parseNotInQuotes() + { + if (_ch == _delimiter) + newField(); + else if (twoCharLineBreak()) + { + newRow(); + nextChar(); + } + else if (oneCharLineBreak()) + newRow(); + else if (isCommentStart()) + skipLine(); + else + saveChar(); + } + + function isCommentStart() + { + if (!_comments) + return false; + + var firstCharOfLine = _i == 0 + || oneCharLineBreak(_i-1) + || twoCharLineBreak(_i-2); + return firstCharOfLine && _input[_i] === _comments; + } + + function skipLine() + { + while (!twoCharLineBreak() + && !oneCharLineBreak() + && _i < _input.length) + { + nextChar(); + } + } + + function saveChar() + { + _data[_rowIdx][_colIdx] += _ch; + } + + function newField() + { + _data[_rowIdx].push(""); + _colIdx = _data[_rowIdx].length - 1; + } + + function newRow() + { + endRow(); + + _lineNum++; + _data.push([]); + _rowIdx = _data.length - 1; + newField(); + } + + function endRow() + { + trimEmptyLastRow(); + if (isFunction(_step)) + { + if (_data[_rowIdx]) + _step(returnable(), self); + clearErrorsAndData(); + } + } + + function trimEmptyLastRow() + { + if (_data[_rowIdx].length == 1 && EMPTY.test(_data[_rowIdx][0])) + { + _data.splice(_rowIdx, 1); + _rowIdx = _data.length - 1; + } + } + + function twoCharLineBreak(i) + { + if (typeof i !== 'number') + i = _i; + return i < _input.length - 1 && + ((_input[i] == "\r" && _input[i+1] == "\n") + || (_input[i] == "\n" && _input[i+1] == "\r")) + } + + function oneCharLineBreak(i) + { + if (typeof i !== 'number') + i = _i; + return _input[i] == "\r" || _input[i] == "\n"; + } + + function quotesEscaped() + { + // Quotes as data cannot be on boundary, for example: ,"", are not escaped quotes + return !quotesOnBoundary() && _i < _input.length - 1 && _input[_i+1] == '"'; + } + + function quotesOnBoundary() + { + return isBoundary(_i-1) || isBoundary(_i+1); + } + + function isBoundary(i) + { + if (typeof i != 'number') + i = _i; + + var ch = _input[i]; + + return (i == -1 || i == _input.length) + || (i < _input.length + && i > -1 + && (ch == _delimiter + || ch == "\r" + || ch == "\n")); + } + + function addError(type, code, msg) + { + _errors.push({ + type: type, + code: code, + message: msg, + line: _lineNum, + row: _rowIdx, + index: _i + }); + } + + function reset(input) + { + _input = input; + _inQuotes = false; + _lineNum = 1; + _i = 0; + clearErrorsAndData(); + _data = [ [""] ]; // starting parsing requires an empty field + _ch = _input[_i]; + } + + function clearErrorsAndData() + { + _data = []; + _errors = []; + _rowIdx = 0; + _colIdx = 0; + } + + function returnable() + { + return { + data: _data, + errors: _errors, + meta: { + lines: _lineNum, + delimiter: _delimiter, + aborted: _aborted + } + }; + } + } + + + + function getScriptPath() + { + var id = "worker" + String(Math.random()).substr(2); + document.write(''); + return document.getElementById(id).previousSibling.src; + } + + function newWorker() + { + if (!Papa.WORKERS_SUPPORTED) + return false; + var w = new global.Worker(SCRIPT_PATH); + w.onmessage = mainThreadReceivedMessage; + w.id = workerIdCounter++; + workers[w.id] = w; + return w; + } + + // Callback when main thread receives a message + function mainThreadReceivedMessage(e) + { + var msg = e.data; + var worker = workers[msg.workerId]; + + if (msg.results && msg.results.data && isFunction(worker.userStep)) + { + for (var i = 0; i < msg.results.data.length; i++) + { + worker.userStep({ + data: [msg.results.data[i]], + errors: msg.results.errors, + meta: msg.results.meta + }); + } + delete msg.results; // free memory ASAP + } + else if (msg.error) + worker.userError(msg.error, msg.file); + + if (msg.finished) + { + if (isFunction(workers[msg.workerId].userComplete)) + workers[msg.workerId].userComplete(msg.results); + workers[msg.workerId].terminate(); + delete workers[msg.workerId]; + } + } + + // Callback when worker thread receives a message + function workerThreadReceivedMessage(e) + { + var msg = e.data; + + if (typeof Papa.WORKER_ID === 'undefined' && msg) + Papa.WORKER_ID = msg.workerId; + + if (typeof msg.input === 'string') + { + global.postMessage({ + workerId: Papa.WORKER_ID, + results: Papa.parse(msg.input, msg.config), + finished: true, + }); + } + else if (msg.input instanceof File) + { + var results = Papa.parse(msg.input, msg.config); + if (results) + global.postMessage({ + workerId: Papa.WORKER_ID, + results: results, + finished: true + }); + } + } + + // Replaces bad config values with good, default ones + function copyAndValidateConfig(origConfig) + { + if (typeof origConfig !== 'object') + origConfig = {}; + + var config = copy(origConfig); + + if (typeof config.delimiter !== 'string' + || config.delimiter.length != 1 + || Papa.BAD_DELIMITERS.indexOf(config.delimiter) > -1) + config.delimiter = DEFAULTS.delimiter; + + if (typeof config.header !== 'boolean') + config.header = DEFAULTS.header; + + if (typeof config.dynamicTyping !== 'boolean') + config.dynamicTyping = DEFAULTS.dynamicTyping; + + if (typeof config.preview !== 'number') + config.preview = DEFAULTS.preview; + + if (typeof config.step !== 'function') + config.step = DEFAULTS.step; + + if (typeof config.complete !== 'function') + config.complete = DEFAULTS.complete; + + if (typeof config.encoding !== 'string') + config.encoding = DEFAULTS.encoding; + + if (typeof config.worker !== 'boolean') + config.worker = DEFAULTS.worker; + + if (typeof config.download !== 'boolean') + config.download = DEFAULTS.download; + + return config; + } + + function copy(obj) + { + if (typeof obj !== 'object') + return obj; + var cpy = obj instanceof Array ? [] : {}; + for (var key in obj) + cpy[key] = copy(obj[key]); + return cpy; + } + + function isFunction(func) + { + return typeof func === 'function'; + } +})(this); \ No newline at end of file diff --git a/papaparse.min.js b/papaparse.min.js new file mode 100644 index 0000000..332d8f5 --- /dev/null +++ b/papaparse.min.js @@ -0,0 +1,6 @@ +/* + Papa Parse + v3.0.0 + https://github.com/mholt/PapaParse +*/ +;(function(e){"use strict";function u(e,n){var r=t?n:g(n);var i=r.worker&&Papa.WORKERS_SUPPORTED;if(i){var s=d();s.userStep=r.step;s.userComplete=r.complete;s.userError=r.error;r.step=b(r.step);r.complete=b(r.complete);r.error=b(r.error);delete r.worker;s.postMessage({input:e,config:r,workerId:s.id})}else{if(typeof e==="string"){if(r.download){var o=new f(r);o.stream(e)}else{var u=new c(r);var a=u.parse(e);if(b(r.complete))r.complete(a);return a}}else if(e instanceof File){if(r.step){var o=new l(r);o.stream(e)}else{var u=new c(r);if(t){var h=new FileReaderSync;var p=h.readAsText(e,r.encoding);return u.parse(p)}else{h=new FileReader;h.onload=function(e){var t=new c(r);var n=t.parse(e.target.result);if(b(r.complete))r.complete(n)};h.readAsText(e,r.encoding)}}}}}function a(t,n){function a(){if(typeof n!=="object")return;if(typeof n.delimiter==="string"&&n.delimiter.length==1&&e.Papa.BAD_DELIMITERS.indexOf(n.delimiter)==-1){o=n.delimiter}if(typeof n.quotes==="boolean")s=n.quotes;if(typeof n.newline==="string")u=n.newline}function f(e){if(typeof e!=="object")return[];var t=[];for(var n in e)t.push(n);return t}function l(e,t){var n="";if(typeof e==="string")e=JSON.parse(e);if(typeof t==="string")t=JSON.parse(t);var r=e instanceof Array&&e.length>0;var i=!(t[0]instanceof Array);if(r){for(var s=0;s0)n+=o;n+=c(e[s])}if(t.length>0)n+=u}for(var a=0;a0)n+=o;var h=r&&i?e[l]:l;n+=c(t[a][h])}if(a-1||t.charAt(0)==" "||t.charAt(t.length-1)==" ";return n?'"'+t+'"':t}function h(e,t){for(var n=0;n-1)return true;return false}var r="";var i=[];var s=false;var o=",";var u="\r\n";a();if(typeof t==="string")t=JSON.parse(t);if(t instanceof Array){if(!t.length||t[0]instanceof Array)return l(null,t);else if(typeof t[0]==="object")return l(f(t[0]),t)}else if(typeof t==="object"){if(typeof t.data==="string")t.data=JSON.parse(t.data);if(t.data instanceof Array){if(!t.fields)t.fields=t.data[0]instanceof Array?t.fields:f(t.data[0]);if(!(t.data[0]instanceof Array)&&typeof t.data[0]!=="object")t.data=[t.data];return l(t.fields,t.data)}}throw"exception: Unable to serialize unrecognized input"}function f(n){n=n||{};if(!n.chunkSize)n.chunkSize=1024*1024*5;var r=0,i=0;var s="";var o="";var u,a;var f=new c(y(n));this.stream=function(l){function c(){u=new XMLHttpRequest;if(!t){u.onload=h;u.onerror=p}u.open("GET",l,!t);if(n.step){var e=r+n.chunkSize-1;if(i&&e>i)e=i;u.setRequestHeader("Range","bytes="+r+"-"+e)}u.send();if(t&&u.status==0)p();else r+=n.chunkSize}function h(){if(u.readyState!=4)return;if(u.status<200||u.status>=400){p();return}s+=o+u.responseText;o="";var i=!n.step||r>d(u);if(!i){var l=s.lastIndexOf("\n");if(l<0)l=s.lastIndexOf("\r");if(l>-1){o=s.substring(l+1);s=s.substring(0,l)}else{a();return}}var c=f.parse(s);s="";if(t){e.postMessage({results:c,workerId:Papa.WORKER_ID,finished:i})}if(i&&b(n.complete))n.complete(c);else if(c&&c.meta.aborted&&b(n.complete))n.complete(c);else if(!i)a()}function p(){if(b(n.error))n.error(u.statusText);else if(t&&n.error){e.postMessage({workerId:Papa.WORKER_ID,error:u.statusText,finished:false})}}function d(e){var t=e.getResponseHeader("Content-Range");return parseInt(t.substr(t.lastIndexOf("/")+1))}if(t){a=function(){c();h()}}else{a=function(){c()}}a()}}function l(n){n=n||{};if(!n.chunkSize)n.chunkSize=1024*1024*10;var r=0;var i="";var s="";var o,u,a;var f=new c(y(n));this.stream=function(u){function l(){if(r=u.size;if(!a){var c=i.lastIndexOf("\n");if(c<0)c=i.lastIndexOf("\r");if(c>-1){s=i.substring(c+1);i=i.substring(0,c)}else{l();return}}var h=f.parse(i);i="";if(t){e.postMessage({results:h,workerId:Papa.WORKER_ID,finished:a})}if(a&&b(n.complete))n.complete(undefined,u);else if(h.meta.aborted&&b(n.complete))n.complete(h,u);else if(!h.meta.paused)l()}function p(){if(b(n.error))n.error(o.error,u);else if(t&&n.error){e.postMessage({workerId:Papa.WORKER_ID,error:o.error,file:u,finished:false})}}var a=u.slice||u.webkitSlice||u.mozSlice;o=new FileReader;o.onload=h;o.onerror=p;l()}}function c(e){function s(){if(i&&n){c("Delimiter","UndetectableDelimiter","Unable to auto-detect delimiting character; defaulted to comma");n=false}if(o())u();return a()}function o(){return e.header&&r.length==0}function u(){if(!i)return;for(var e=0;o()&&e=r.length){if(!n["__parsed_extra"])n["__parsed_extra"]=[];n["__parsed_extra"].push(i.data[t][s])}n[r[s]]=i.data[t][s]}}if(e.header){i.data[t]=n;if(s>r.length)c("FieldMismatch","TooManyFields","Too many fields: expected "+r.length+" fields but parsed "+s,t);else if(s1){a+=Math.abs(p-s);s=p}}f/=l.data.length;if((typeof i==="undefined"||a1.99){i=a;r=u}}e.delimiter=r;return{successful:!!r,bestDelimiter:r}}function l(e){var n=t.test(e);return n?parseFloat(e):e}function c(e,t,n,r){i.errors.push({type:e,code:t,message:n,row:r})}var t=/^\s*-?(\d*\.?\d+|\d+\.?\d*)(e[-+]?\d+)?\s*$/i;var n;var r=[];var i={data:[],errors:[],meta:{}};e=y(e);this.parse=function(t){n=false;if(!e.delimiter){var r=f(t);if(r.successful)e.delimiter=r.bestDelimiter;else{n=true;e.delimiter=","}i.meta.delimiter=e.delimiter}if(b(e.step)){var u=e.step;e.step=function(e,t){i=e;if(o())s();else u(s(),t)}}i=(new h(e)).parse(t);return s()}}function h(e){function w(){while(l0&&v>=a)break;if(y)return S();if(f=='"')x();else if(c)T();else N();E()}return S()}function E(){l++;f=r[l]}function S(){if(g)F("Abort","ParseAbort","Parsing was aborted by the user's step function");if(c)F("Quotes","MissingQuotes","Unescaped or mismatched quotes");M();if(!b(o))return R()}function x(){if(B()&&!H())c=!c;else{L();if(c&&H())l++;else F("Quotes","UnexpectedQuotes","Unexpected quotes")}}function T(){L()}function N(){if(f==i)A();else if(D()){O();E()}else if(P())O();else if(C())k();else L()}function C(){if(!s)return false;var e=l==0||P(l-1)||D(l-2);return e&&r[l]===s}function k(){while(!D()&&!P()&&l-1&&(t==i||t=="\r"||t=="\n")}function F(e,t,n){d.push({type:e,code:t,message:n,line:h,row:v,index:l})}function I(e){r=e;c=false;h=1;l=0;q();p=[[""]];f=r[l]}function q(){p=[];d=[];v=0;m=0}function R(){return{data:p,errors:d,meta:{lines:h,delimiter:i,aborted:g}}}var t=this;var n=/^\s*$/;var r;var i;var s;var o;var u;var a;var f;var l;var c;var h;var p;var d;var v;var m;var g=false;var y=false;e=e||{};i=e.delimiter;s=e.comments;o=e.step;a=e.preview;if(typeof i!=="string"||i.length!=1||Papa.BAD_DELIMITERS.indexOf(i)>-1)i=",";if(s===true)s="#";else if(typeof s!=="string"||s.length!=1||Papa.BAD_DELIMITERS.indexOf(s)>-1||s==i)s=false;this.parse=function(e){if(typeof e!=="string")throw"Input must be a string";I(e);return w()};this.abort=function(){g=true}}function p(){var e="worker"+String(Math.random()).substr(2);document.write('');return document.getElementById(e).previousSibling.src}function d(){if(!Papa.WORKERS_SUPPORTED)return false;var t=new e.Worker(n);t.onmessage=v;t.id=i++;r[t.id]=t;return t}function v(e){var t=e.data;var n=r[t.workerId];if(t.results&&t.results.data&&b(n.userStep)){for(var i=0;i-1)t.delimiter=s.delimiter;if(typeof t.header!=="boolean")t.header=s.header;if(typeof t.dynamicTyping!=="boolean")t.dynamicTyping=s.dynamicTyping;if(typeof t.preview!=="number")t.preview=s.preview;if(typeof t.step!=="function")t.step=s.step;if(typeof t.complete!=="function")t.complete=s.complete;if(typeof t.encoding!=="string")t.encoding=s.encoding;if(typeof t.worker!=="boolean")t.worker=s.worker;if(typeof t.download!=="boolean")t.download=s.download;return t}function y(e){if(typeof e!=="object")return e;var t=e instanceof Array?[]:{};for(var n in e)t[n]=y(e[n]);return t}function b(e){return typeof e==="function"}var t=!e.document,n;var r={},i=0;var s={delimiter:"",header:false,dynamicTyping:false,preview:0,step:undefined,encoding:"",worker:false,comments:false,complete:undefined,download:false};e.Papa={};e.Papa.parse=u;e.Papa.unparse=a;e.Papa.RECORD_SEP=String.fromCharCode(30);e.Papa.UNIT_SEP=String.fromCharCode(31);e.Papa.BYTE_ORDER_MARK="";e.Papa.BAD_DELIMITERS=["\r","\n",'"',e.Papa.BYTE_ORDER_MARK];e.Papa.WORKERS_SUPPORTED=!!e.Worker;e.Papa.Parser=h;e.Papa.ParserHandle=c;e.Papa.NetworkStreamer=f;e.Papa.FileStreamer=l;if(e.jQuery){var o=e.jQuery;o.fn.parse=function(t){function i(){if(r.length==0){if(b(t.complete))t.complete();return}var e=r[0];if(b(t.before)){var n=t.before(e.file,e.inputElem);if(typeof n==="object"){if(n.action=="abort"){s("AbortError",e.file,e.inputElem,n.reason);return}else if(n.action=="skip"){u();return}else if(typeof n.config==="object")e.instanceConfig=o.extend(e.instanceConfig,n.config)}else if(n=="skip"){u();return}}var i=e.instanceConfig.complete;e.instanceConfig.complete=function(t){if(b(i))i(t,e.file,e.inputElem);u()};Papa.parse(e.file,e.instanceConfig)}function s(e,n,r,i){if(b(t.error))t.error({name:e},n,r,i)}function u(){r.splice(0,1);i()}var n=t.config||{};var r=[];this.each(function(t){var i=o(this).prop("tagName").toUpperCase()=="INPUT"&&o(this).attr("type").toLowerCase()=="file"&&e.FileReader;if(!i||!this.files||this.files.length==0)return true;for(var s=0;s=1.6.0" } diff --git a/player/player.css b/player/player.css new file mode 100644 index 0000000..1683f88 --- /dev/null +++ b/player/player.css @@ -0,0 +1,1041 @@ +body { + font-family: 'Source Sans Pro', sans-serif; +} + +h1 { + text-align: center; +} + +textarea, +button { + font-size: 14px; +} + +textarea { + box-sizing: border-box; + font: 14px/1.25em Menlo, Monaco, 'Courier New', monospace; + width: 100%; + padding: 10px; + max-width: 900px; + height: 200px; +} + +button { + padding: 10px 50px; + font-size: 20px; +} + +label { + display: block; +} + + +.text-center { + text-align: center; +} + + + + + + + + + +/* ============================================ */ +/* This file has a mobile-to-desktop breakpoint */ +/* ============================================ */ +@media screen and (max-width: 400px) { + @-ms-viewport { + width: 320px; + } +} +.clear { + clear: both; + display: block; + overflow: hidden; + visibility: hidden; + width: 0; + height: 0; +} + +.grid-container:before, .clearfix:before, +.grid-container:after, +.clearfix:after { + content: "."; + display: block; + overflow: hidden; + visibility: hidden; + font-size: 0; + line-height: 0; + width: 0; + height: 0; +} + +.grid-container:after, .clearfix:after { + clear: both; +} + +.grid-container { + margin-left: auto; + margin-right: auto; + max-width: 1200px; + padding-left: 10px; + padding-right: 10px; +} + +.grid-5, .mobile-grid-5, .grid-10, .mobile-grid-10, .grid-15, .mobile-grid-15, .grid-20, .mobile-grid-20, .grid-25, .mobile-grid-25, .grid-30, .mobile-grid-30, .grid-35, .mobile-grid-35, .grid-40, .mobile-grid-40, .grid-45, .mobile-grid-45, .grid-50, .mobile-grid-50, .grid-55, .mobile-grid-55, .grid-60, .mobile-grid-60, .grid-65, .mobile-grid-65, .grid-70, .mobile-grid-70, .grid-75, .mobile-grid-75, .grid-80, .mobile-grid-80, .grid-85, .mobile-grid-85, .grid-90, .mobile-grid-90, .grid-95, .mobile-grid-95, .grid-100, .mobile-grid-100, .grid-33, .mobile-grid-33, .grid-66, .mobile-grid-66 { + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; + box-sizing: border-box; + padding-left: 10px; + padding-right: 10px; +} + +.grid-parent { + padding-left: 0; + padding-right: 0; +} + +@media screen and (max-width: 767px) { + .mobile-grid-100:before, + .mobile-grid-100:after { + content: "."; + display: block; + overflow: hidden; + visibility: hidden; + font-size: 0; + line-height: 0; + width: 0; + height: 0; + } + + .mobile-grid-100:after { + clear: both; + } + + .mobile-push-5, .mobile-pull-5, .mobile-push-10, .mobile-pull-10, .mobile-push-15, .mobile-pull-15, .mobile-push-20, .mobile-pull-20, .mobile-push-25, .mobile-pull-25, .mobile-push-30, .mobile-pull-30, .mobile-push-35, .mobile-pull-35, .mobile-push-40, .mobile-pull-40, .mobile-push-45, .mobile-pull-45, .mobile-push-50, .mobile-pull-50, .mobile-push-55, .mobile-pull-55, .mobile-push-60, .mobile-pull-60, .mobile-push-65, .mobile-pull-65, .mobile-push-70, .mobile-pull-70, .mobile-push-75, .mobile-pull-75, .mobile-push-80, .mobile-pull-80, .mobile-push-85, .mobile-pull-85, .mobile-push-90, .mobile-pull-90, .mobile-push-95, .mobile-pull-95, .mobile-push-33, .mobile-pull-33, .mobile-push-66, .mobile-pull-66 { + position: relative; + } + + .hide-on-mobile { + display: none !important; + } + + .mobile-grid-5 { + float: left; + width: 5%; + } + + .mobile-prefix-5 { + margin-left: 5%; + } + + .mobile-suffix-5 { + margin-right: 5%; + } + + .mobile-push-5 { + left: 5%; + } + + .mobile-pull-5 { + left: -5%; + } + + .mobile-grid-10 { + float: left; + width: 10%; + } + + .mobile-prefix-10 { + margin-left: 10%; + } + + .mobile-suffix-10 { + margin-right: 10%; + } + + .mobile-push-10 { + left: 10%; + } + + .mobile-pull-10 { + left: -10%; + } + + .mobile-grid-15 { + float: left; + width: 15%; + } + + .mobile-prefix-15 { + margin-left: 15%; + } + + .mobile-suffix-15 { + margin-right: 15%; + } + + .mobile-push-15 { + left: 15%; + } + + .mobile-pull-15 { + left: -15%; + } + + .mobile-grid-20 { + float: left; + width: 20%; + } + + .mobile-prefix-20 { + margin-left: 20%; + } + + .mobile-suffix-20 { + margin-right: 20%; + } + + .mobile-push-20 { + left: 20%; + } + + .mobile-pull-20 { + left: -20%; + } + + .mobile-grid-25 { + float: left; + width: 25%; + } + + .mobile-prefix-25 { + margin-left: 25%; + } + + .mobile-suffix-25 { + margin-right: 25%; + } + + .mobile-push-25 { + left: 25%; + } + + .mobile-pull-25 { + left: -25%; + } + + .mobile-grid-30 { + float: left; + width: 30%; + } + + .mobile-prefix-30 { + margin-left: 30%; + } + + .mobile-suffix-30 { + margin-right: 30%; + } + + .mobile-push-30 { + left: 30%; + } + + .mobile-pull-30 { + left: -30%; + } + + .mobile-grid-35 { + float: left; + width: 35%; + } + + .mobile-prefix-35 { + margin-left: 35%; + } + + .mobile-suffix-35 { + margin-right: 35%; + } + + .mobile-push-35 { + left: 35%; + } + + .mobile-pull-35 { + left: -35%; + } + + .mobile-grid-40 { + float: left; + width: 40%; + } + + .mobile-prefix-40 { + margin-left: 40%; + } + + .mobile-suffix-40 { + margin-right: 40%; + } + + .mobile-push-40 { + left: 40%; + } + + .mobile-pull-40 { + left: -40%; + } + + .mobile-grid-45 { + float: left; + width: 45%; + } + + .mobile-prefix-45 { + margin-left: 45%; + } + + .mobile-suffix-45 { + margin-right: 45%; + } + + .mobile-push-45 { + left: 45%; + } + + .mobile-pull-45 { + left: -45%; + } + + .mobile-grid-50 { + float: left; + width: 50%; + } + + .mobile-prefix-50 { + margin-left: 50%; + } + + .mobile-suffix-50 { + margin-right: 50%; + } + + .mobile-push-50 { + left: 50%; + } + + .mobile-pull-50 { + left: -50%; + } + + .mobile-grid-55 { + float: left; + width: 55%; + } + + .mobile-prefix-55 { + margin-left: 55%; + } + + .mobile-suffix-55 { + margin-right: 55%; + } + + .mobile-push-55 { + left: 55%; + } + + .mobile-pull-55 { + left: -55%; + } + + .mobile-grid-60 { + float: left; + width: 60%; + } + + .mobile-prefix-60 { + margin-left: 60%; + } + + .mobile-suffix-60 { + margin-right: 60%; + } + + .mobile-push-60 { + left: 60%; + } + + .mobile-pull-60 { + left: -60%; + } + + .mobile-grid-65 { + float: left; + width: 65%; + } + + .mobile-prefix-65 { + margin-left: 65%; + } + + .mobile-suffix-65 { + margin-right: 65%; + } + + .mobile-push-65 { + left: 65%; + } + + .mobile-pull-65 { + left: -65%; + } + + .mobile-grid-70 { + float: left; + width: 70%; + } + + .mobile-prefix-70 { + margin-left: 70%; + } + + .mobile-suffix-70 { + margin-right: 70%; + } + + .mobile-push-70 { + left: 70%; + } + + .mobile-pull-70 { + left: -70%; + } + + .mobile-grid-75 { + float: left; + width: 75%; + } + + .mobile-prefix-75 { + margin-left: 75%; + } + + .mobile-suffix-75 { + margin-right: 75%; + } + + .mobile-push-75 { + left: 75%; + } + + .mobile-pull-75 { + left: -75%; + } + + .mobile-grid-80 { + float: left; + width: 80%; + } + + .mobile-prefix-80 { + margin-left: 80%; + } + + .mobile-suffix-80 { + margin-right: 80%; + } + + .mobile-push-80 { + left: 80%; + } + + .mobile-pull-80 { + left: -80%; + } + + .mobile-grid-85 { + float: left; + width: 85%; + } + + .mobile-prefix-85 { + margin-left: 85%; + } + + .mobile-suffix-85 { + margin-right: 85%; + } + + .mobile-push-85 { + left: 85%; + } + + .mobile-pull-85 { + left: -85%; + } + + .mobile-grid-90 { + float: left; + width: 90%; + } + + .mobile-prefix-90 { + margin-left: 90%; + } + + .mobile-suffix-90 { + margin-right: 90%; + } + + .mobile-push-90 { + left: 90%; + } + + .mobile-pull-90 { + left: -90%; + } + + .mobile-grid-95 { + float: left; + width: 95%; + } + + .mobile-prefix-95 { + margin-left: 95%; + } + + .mobile-suffix-95 { + margin-right: 95%; + } + + .mobile-push-95 { + left: 95%; + } + + .mobile-pull-95 { + left: -95%; + } + + .mobile-grid-33 { + float: left; + width: 33.33333%; + } + + .mobile-prefix-33 { + margin-left: 33.33333%; + } + + .mobile-suffix-33 { + margin-right: 33.33333%; + } + + .mobile-push-33 { + left: 33.33333%; + } + + .mobile-pull-33 { + left: -33.33333%; + } + + .mobile-grid-66 { + float: left; + width: 66.66667%; + } + + .mobile-prefix-66 { + margin-left: 66.66667%; + } + + .mobile-suffix-66 { + margin-right: 66.66667%; + } + + .mobile-push-66 { + left: 66.66667%; + } + + .mobile-pull-66 { + left: -66.66667%; + } + + .mobile-grid-100 { + clear: both; + width: 100%; + } +} +@media screen and (min-width: 768px) { + .grid-100:before, + .grid-100:after { + content: "."; + display: block; + overflow: hidden; + visibility: hidden; + font-size: 0; + line-height: 0; + width: 0; + height: 0; + } + + .grid-100:after { + clear: both; + } + + .push-5, .pull-5, .push-10, .pull-10, .push-15, .pull-15, .push-20, .pull-20, .push-25, .pull-25, .push-30, .pull-30, .push-35, .pull-35, .push-40, .pull-40, .push-45, .pull-45, .push-50, .pull-50, .push-55, .pull-55, .push-60, .pull-60, .push-65, .pull-65, .push-70, .pull-70, .push-75, .pull-75, .push-80, .pull-80, .push-85, .pull-85, .push-90, .pull-90, .push-95, .pull-95, .push-33, .pull-33, .push-66, .pull-66 { + position: relative; + } + + .hide-on-desktop { + display: none !important; + } + + .grid-5 { + float: left; + width: 5%; + } + + .prefix-5 { + margin-left: 5%; + } + + .suffix-5 { + margin-right: 5%; + } + + .push-5 { + left: 5%; + } + + .pull-5 { + left: -5%; + } + + .grid-10 { + float: left; + width: 10%; + } + + .prefix-10 { + margin-left: 10%; + } + + .suffix-10 { + margin-right: 10%; + } + + .push-10 { + left: 10%; + } + + .pull-10 { + left: -10%; + } + + .grid-15 { + float: left; + width: 15%; + } + + .prefix-15 { + margin-left: 15%; + } + + .suffix-15 { + margin-right: 15%; + } + + .push-15 { + left: 15%; + } + + .pull-15 { + left: -15%; + } + + .grid-20 { + float: left; + width: 20%; + } + + .prefix-20 { + margin-left: 20%; + } + + .suffix-20 { + margin-right: 20%; + } + + .push-20 { + left: 20%; + } + + .pull-20 { + left: -20%; + } + + .grid-25 { + float: left; + width: 25%; + } + + .prefix-25 { + margin-left: 25%; + } + + .suffix-25 { + margin-right: 25%; + } + + .push-25 { + left: 25%; + } + + .pull-25 { + left: -25%; + } + + .grid-30 { + float: left; + width: 30%; + } + + .prefix-30 { + margin-left: 30%; + } + + .suffix-30 { + margin-right: 30%; + } + + .push-30 { + left: 30%; + } + + .pull-30 { + left: -30%; + } + + .grid-35 { + float: left; + width: 35%; + } + + .prefix-35 { + margin-left: 35%; + } + + .suffix-35 { + margin-right: 35%; + } + + .push-35 { + left: 35%; + } + + .pull-35 { + left: -35%; + } + + .grid-40 { + float: left; + width: 40%; + } + + .prefix-40 { + margin-left: 40%; + } + + .suffix-40 { + margin-right: 40%; + } + + .push-40 { + left: 40%; + } + + .pull-40 { + left: -40%; + } + + .grid-45 { + float: left; + width: 45%; + } + + .prefix-45 { + margin-left: 45%; + } + + .suffix-45 { + margin-right: 45%; + } + + .push-45 { + left: 45%; + } + + .pull-45 { + left: -45%; + } + + .grid-50 { + float: left; + width: 50%; + } + + .prefix-50 { + margin-left: 50%; + } + + .suffix-50 { + margin-right: 50%; + } + + .push-50 { + left: 50%; + } + + .pull-50 { + left: -50%; + } + + .grid-55 { + float: left; + width: 55%; + } + + .prefix-55 { + margin-left: 55%; + } + + .suffix-55 { + margin-right: 55%; + } + + .push-55 { + left: 55%; + } + + .pull-55 { + left: -55%; + } + + .grid-60 { + float: left; + width: 60%; + } + + .prefix-60 { + margin-left: 60%; + } + + .suffix-60 { + margin-right: 60%; + } + + .push-60 { + left: 60%; + } + + .pull-60 { + left: -60%; + } + + .grid-65 { + float: left; + width: 65%; + } + + .prefix-65 { + margin-left: 65%; + } + + .suffix-65 { + margin-right: 65%; + } + + .push-65 { + left: 65%; + } + + .pull-65 { + left: -65%; + } + + .grid-70 { + float: left; + width: 70%; + } + + .prefix-70 { + margin-left: 70%; + } + + .suffix-70 { + margin-right: 70%; + } + + .push-70 { + left: 70%; + } + + .pull-70 { + left: -70%; + } + + .grid-75 { + float: left; + width: 75%; + } + + .prefix-75 { + margin-left: 75%; + } + + .suffix-75 { + margin-right: 75%; + } + + .push-75 { + left: 75%; + } + + .pull-75 { + left: -75%; + } + + .grid-80 { + float: left; + width: 80%; + } + + .prefix-80 { + margin-left: 80%; + } + + .suffix-80 { + margin-right: 80%; + } + + .push-80 { + left: 80%; + } + + .pull-80 { + left: -80%; + } + + .grid-85 { + float: left; + width: 85%; + } + + .prefix-85 { + margin-left: 85%; + } + + .suffix-85 { + margin-right: 85%; + } + + .push-85 { + left: 85%; + } + + .pull-85 { + left: -85%; + } + + .grid-90 { + float: left; + width: 90%; + } + + .prefix-90 { + margin-left: 90%; + } + + .suffix-90 { + margin-right: 90%; + } + + .push-90 { + left: 90%; + } + + .pull-90 { + left: -90%; + } + + .grid-95 { + float: left; + width: 95%; + } + + .prefix-95 { + margin-left: 95%; + } + + .suffix-95 { + margin-right: 95%; + } + + .push-95 { + left: 95%; + } + + .pull-95 { + left: -95%; + } + + .grid-33 { + float: left; + width: 33.33333%; + } + + .prefix-33 { + margin-left: 33.33333%; + } + + .suffix-33 { + margin-right: 33.33333%; + } + + .push-33 { + left: 33.33333%; + } + + .pull-33 { + left: -33.33333%; + } + + .grid-66 { + float: left; + width: 66.66667%; + } + + .prefix-66 { + margin-left: 66.66667%; + } + + .suffix-66 { + margin-right: 66.66667%; + } + + .push-66 { + left: 66.66667%; + } + + .pull-66 { + left: -66.66667%; + } + + .grid-100 { + clear: both; + width: 100%; + } +} \ No newline at end of file diff --git a/player/player.html b/player/player.html new file mode 100644 index 0000000..a0804ab --- /dev/null +++ b/player/player.html @@ -0,0 +1,53 @@ + + + + Papa Parse Player + + + + + + + +

Papa Parse Player

+ +
+ +
+ + + + + + + + + +
+ +
+ + + +
+ or +
+ + + +

+ + + +

+ + Open the Console in your browser's inspector tools to see results. +
+ +
+ + \ No newline at end of file diff --git a/player/player.js b/player/player.js new file mode 100644 index 0000000..9af6397 --- /dev/null +++ b/player/player.js @@ -0,0 +1,70 @@ +var stepped = 0; +var start, end; + +$(function() +{ + $('#submit').click(function() + { + stepped = 0; + var txt = $('#input').val(); + var files = $('#files')[0].files; + var config = buildConfig(); + + if (files.length > 0) + { + start = performance.now(); + + $('#files').parse({ + config: config, + before: function(file, inputElem) + { + console.log("Parsing file:", file); + }, + complete: function() + { + console.log("Done with all files."); + } + }); + } + else + { + start = performance.now(); + var results = Papa.parse(txt, config); + console.log("Synchronous parse results:", results); + } + }); + + $('#insert-tab').click(function() + { + $('#delimiter').val('\t'); + }); +}); + + + +function buildConfig() +{ + return { + delimiter: $('#delimiter').val(), + header: $('#header').prop('checked'), + dynamicTyping: $('#dynamicTyping').prop('checked'), + preview: parseInt($('#preview').val() || 0), + step: $('#stream').prop('checked') ? stepFn : undefined, + encoding: $('#encoding').val(), + worker: $('#worker').prop('checked'), + comments: $('#comments').val(), + complete: completeFn, + download: $('#download').prop('checked') + }; +} + +function stepFn(results, parser) +{ + stepped++; +} + +function completeFn() +{ + end = performance.now(); + console.log("Finished input. Time:", end-start, arguments); +} \ No newline at end of file diff --git a/tests.html b/tests.html deleted file mode 100644 index acf8998..0000000 --- a/tests.html +++ /dev/null @@ -1,127 +0,0 @@ - - - - Parser Tests - - - - - - - SUMMARY -   - - PASS -   - - FAIL -

- - - - - - - - -
InputConfigExpectedActual
-
- - \ No newline at end of file diff --git a/tests.js b/tests.js deleted file mode 100644 index 713a387..0000000 --- a/tests.js +++ /dev/null @@ -1,693 +0,0 @@ -var passCount = 0, failCount = 0; -var passing = "passing"; -var failing = "failing"; - -var recordSep = String.fromCharCode(30); -var unitSep = String.fromCharCode(31); - -var resultSet1 = [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": 2, - "F3": "V3" - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: ",", header: false, dynamicTyping: true }, - expected: { - "results": [ - [ - "F1", - "F2", - "F3" - ], - [ - "V1", - 2, - "V3" - ] - ], - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: ",", header: false, dynamicTyping: false }, - expected: { - "results": [ - [ - "F1", - "F2", - "F3" - ], - [ - "V1", - "2", - "V3" - ] - ], - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: ",", header: true, dynamicTyping: false }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": "2", - "F3": "V3" - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: "", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": 2, - "F3": "V3" - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, -]; - -var tests = [ - { - input: "F1,F2,F3\nV1,2,V3", - cases: resultSet1 - }, - { - input: "F1,F2,F3\r\nV1,2,V3", - cases: resultSet1 - }, - { - input: "F1,\"F2\",F3\r\nV1,2,\"V3\"", - cases: resultSet1 - }, - { - input: "F1,F2,F3\n\nV1,2,V3", - cases: resultSet1 - }, - { - input: "F1,F2,F3\r\n\r\nV1,2,V3", - cases: resultSet1 - }, - { - input: "F1,F2,F3\n\rV1,2,V3", - cases: resultSet1 - }, - { - input: "F1,F2,F3\rV1,2,V3", - cases: resultSet1 - }, - { - input: "F1,F2,F3\r\n \r\nV1,2,V3", - cases: resultSet1 - }, - { - input: "\r\nF1,F2,F3\r\nV1,2,V3", - cases: resultSet1 - }, - { - input: 'F1,F2,"F3"\n"V1","2",V3', - cases: resultSet1 - }, - { - input: "F1,F2,F3\nV1,2,V3\nV4,V5,V6", - cases: [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": 2, - "F3": "V3" - }, - { - "F1": "V4", - "F2": "V5", - "F3": "V6" - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: ",", header: false, dynamicTyping: true }, - expected: { - "results": [ - [ - "F1", - "F2", - "F3" - ], - [ - "V1", - 2, - "V3" - ], - [ - "V4", - "V5", - "V6" - ] - ], - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: ",", header: false, dynamicTyping: false }, - expected: { - "results": [ - [ - "F1", - "F2", - "F3" - ], - [ - "V1", - "2", - "V3" - ], - [ - "V4", - "V5", - "V6" - ] - ], - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - }, - { - config: { delimiter: ",", header: true, dynamicTyping: false }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": "2", - "F3": "V3" - }, - { - "F1": "V4", - "F2": "V5", - "F3": "V6" - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: "F1,F2,F3\n,2,V3\nV4,V5,V6", - cases: [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "", - "F2": 2, - "F3": "V3" - }, - { - "F1": "V4", - "F2": "V5", - "F3": "V6" - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: "F1,F2,F3\n,2,V3,V4\nV5,V6,V7", - cases: [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "", - "F2": 2, - "F3": "V3", - "__parsed_extra": [ - "V4" - ] - }, - { - "F1": "V5", - "F2": "V6", - "F3": "V7" - } - ] - }, - "errors": { - "0": [ - { - "type": "FieldMismatch", - "code": "TooManyFields", - "message": "Too many fields: expected 3 fields but parsed 4", - "line": 2, - "row": 0, - "index": 17 - } - ], - "length": 1 - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: "F1,F2,F3\nV1,2.0,-3.01, V4\n\rV5,\"V\n6\",V7\r,,", - cases: [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": 2, - "F3": -3.01, - "__parsed_extra": [ - " V4" - ] - }, - { - "F1": "V5", - "F2": "V\n6", - "F3": "V7" - }, - { - "F1": "", - "F2": "", - "F3": "" - } - ] - }, - "errors": { - "0": [ - { - "type": "FieldMismatch", - "code": "TooManyFields", - "message": "Too many fields: expected 3 fields but parsed 4", - "line": 2, - "row": 0, - "index": 25 - } - ], - "length": 1 - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: "F1,F2,F3\nV1,V2,V3\nV5,\"V6,V7", - cases: [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": "V1", - "F2": "V2", - "F3": "V3" - }, - { - "F1": "V5", - "F2": "V6,V7" - } - ] - }, - "errors": { - "1": [ - { - "type": "FieldMismatch", - "code": "TooFewFields", - "message": "Too few fields: expected 3 fields but parsed 2", - "line": 3, - "row": 1, - "index": 27 - }, - { - "type": "Quotes", - "code": "MissingQuotes", - "message": "Unescaped or mismatched quotes", - "line": 3, - "row": 1, - "index": 27 - } - ], - "length": 2 - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: "F1,F2,F3\n2,-2, 2\n 2. ,.2, .2 \n-2., -2.0e-5, -.4 ", - cases: [ - { - config: { delimiter: ",", header: true, dynamicTyping: true }, - expected: { - "results": { - "fields": [ - "F1", - "F2", - "F3" - ], - "rows": [ - { - "F1": 2, - "F2": -2, - "F3": 2 - }, - { - "F1": 2, - "F2": 0.2, - "F3": 0.2 - }, - { - "F1": -2, - "F2": -0.00002, - "F3": -0.4 - } - ] - }, - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: "F1\nV1\nV2\nV3\nV4", - cases: [ - { - config: { delimiter: "", header: false, dynamicTyping: false }, - expected: { - "results": [ - [ - "F1" - ], - [ - "V1" - ], - [ - "V2" - ], - [ - "V3" - ], - [ - "V4" - ] - ], - "errors": { - "length": 1, - "config": [ - { - "type": "Delimiter", - "code": "UndetectableDelimiter", - "message": "Unable to auto-detect delimiting character; defaulted to comma", - "line": 1, - "row": 0, - "index": 0 - } - ] - }, - "meta": { - "delimiter": "," - } - } - } - ] - }, - { - input: ["F1","F2","F3\r\nV1","V2","V3"].join(recordSep), - cases: [ - { - config: { delimiter: "", header: false, dynamicTyping: false }, - expected: { - "results": [ - [ - "F1", - "F2", - "F3" - ], - [ - "V1", - "V2", - "V3" - ], - ], - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "\u001e" - } - } - } - ] - }, - { - input: ["F1","F2","F3\r\nV1","V2","V3"].join(unitSep), - cases: [ - { - config: { delimiter: "", header: false, dynamicTyping: false }, - expected: { - "results": [ - [ - "F1", - "F2", - "F3" - ], - [ - "V1", - "V2", - "V3" - ], - ], - "errors": { - "length": 0 - }, - "meta": { - "delimiter": "\u001f" - } - } - } - ] - } -]; - - - - - - - -$(function() -{ - var counter = 0; - for (var i = 0; i < tests.length; i++) - { - var test = tests[i]; - var input = test.input; - for (var j = 0; j < test.cases.length; j++) - { - counter++; - var testCase = test.cases[j]; - var actual = doTest(input, testCase.config); - var status = equal(actual, testCase.expected) ? passing : failing; - render(input, testCase.expected, actual, testCase.config, counter, status); - } - } - - $('#pass-count').text(passCount); - $('#fail-count').text(failCount); -}); - -function doTest(input, config) -{ - // try - // { - return $.parse(input, config); - // } - // catch (e) - // { - // return {exception: e.message, error: e, note: "See console to inspect stack"}; - // } -} - -function render(input, expected, actual, config, count, status) -{ - if (status == passing) - passCount++; - else - { - console.log("TEST " + count +" FAILED."); - console.log(" Expected:", expected); - console.log(" Actual:", actual); - console.log(" Config:", config); - failCount++; - } - - var html = '' + - ''+count+'' + - '
'+string(input)+'
' + - '
'+string(config)+'
' + - '
'+string(expected)+'
' + - '
'+string(actual)+'
' + - ''; - $('#results').append(html); -} - -function string(obj) -{ - return typeof obj === "string" ? obj : JSON.stringify(obj, undefined, 2); -} - -function equal(actual, expected) -{ - return string(actual) === string(expected); -} \ No newline at end of file diff --git a/tests/test-cases.js b/tests/test-cases.js new file mode 100644 index 0000000..4624ed4 --- /dev/null +++ b/tests/test-cases.js @@ -0,0 +1,401 @@ +// TODO: Add tests for unparse: +// If fields is omitted, write a CSV string without a header row +// If delimiter is omitted, choose comma by default +// If data is omitted, do nothing... maybe if fields IS specified, write just the header row? + +var RECORD_SEP = String.fromCharCode(30); +var UNIT_SEP = String.fromCharCode(31); + +var TESTS = [ + { + input: 'A,b,c', + description: "One row", + expected: { + data: [['A', 'b', 'c']], + errors: [] + } + }, + { + input: 'A,b,c\r\nd,E,f', + description: "Two rows", + expected: { + data: [['A', 'b', 'c'], ['d', 'E', 'f']], + errors: [] + } + }, + { + input: 'A,b,c\rd,E,f', + description: "Two rows, just \\r", + expected: { + data: [['A', 'b', 'c'], ['d', 'E', 'f']], + errors: [] + } + }, + { + input: 'A,b,c\nd,E,f', + description: "Two rows, just \\n", + expected: { + data: [['A', 'b', 'c'], ['d', 'E', 'f']], + errors: [] + } + }, + { + input: 'a, b ,c', + description: "Whitespace at edges of unquoted field", + notes: "Extra whitespace should graciously be preserved", + expected: { + data: [['a', ' b ', 'c']], + errors: [] + } + }, + { + input: 'A,"B",C', + description: "Quoted field", + expected: { + data: [['A', 'B', 'C']], + errors: [] + } + }, + { + input: 'A," B ",C', + description: "Quoted field with extra whitespace on edges", + expected: { + data: [['A', ' B ', 'C']], + errors: [] + } + }, + { + input: 'A,"B,B",C', + description: "Quoted field with delimiter", + expected: { + data: [['A', 'B,B', 'C']], + errors: [] + } + }, + { + input: 'A,"B\r\nB",C', + description: "Quoted field with \\r\\n", + expected: { + data: [['A', 'B\r\nB', 'C']], + errors: [] + } + }, + { + input: 'A,"B\rB",C', + description: "Quoted field with \\r", + expected: { + data: [['A', 'B\rB', 'C']], + errors: [] + } + }, + { + input: 'A,"B\nB",C', + description: "Quoted field with \\n", + expected: { + data: [['A', 'B\nB', 'C']], + errors: [] + } + }, + { + input: 'A,"B""B""B",C', + description: "Quoted field with escaped quotes", + expected: { + data: [['A', 'B"B"B', 'C']], + errors: [] + } + }, + { + input: 'A,"""B""",C', + description: "Quoted field with escaped quotes at boundaries", + expected: { + data: [['A', '"B"', 'C']], + errors: [] + } + }, + { + input: 'A, "B" ,C', + description: "Quoted field with whitespace around quotes", + notes: "This is malformed input, but it should be parsed gracefully (with errors)", + expected: { + data: [['A', ' "B" ', 'C']], + errors: [ + {"type": "Quotes", "code": "UnexpectedQuotes", "message": "Unexpected quotes", "line": 1, "row": 0, "index": 3}, + {"type": "Quotes", "code": "UnexpectedQuotes", "message": "Unexpected quotes", "line": 1, "row": 0, "index": 5} + ] + } + }, + { + input: 'a\tb\tc\r\nd\te\tf', + config: { delimiter: "\t" }, + description: "Tab delimiter", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a|b|c\r\nd|e|f', + config: { delimiter: "|" }, + description: "Pipe delimiter", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a'+RECORD_SEP+'b'+RECORD_SEP+'c\r\nd'+RECORD_SEP+'e'+RECORD_SEP+'f', + config: { delimiter: RECORD_SEP }, + description: "ASCII 30 delimiter", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a'+UNIT_SEP+'b'+UNIT_SEP+'c\r\nd'+UNIT_SEP+'e'+UNIT_SEP+'f', + config: { delimiter: UNIT_SEP }, + description: "ASCII 31 delimiter", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c', + config: { delimiter: "DELIM" }, + description: "Bad delimiter", + notes: "Should silently default to comma", + expected: { + data: [['a', 'b', 'c']], + errors: [] + } + }, + { + input: '# Comment!\r\na,b,c', + config: { comments: true }, + description: "Commented line at beginning (comments: true)", + expected: { + data: [['a', 'b', 'c']], + errors: [] + } + }, + { + input: 'a,b,c\r\n# Comment\r\nd,e,f', + config: { comments: true }, + description: "Commented line in middle (comments: true)", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\n# Comment', + config: { comments: true }, + description: "Commented line at end (comments: true)", + expected: { + data: [['a', 'b', 'c']], + errors: [] + } + }, + { + input: 'a,b,c\r\n!Comment goes here\r\nd,e,f', + config: { comments: '!' }, + description: "Comment with non-default character (comments: '!')", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\n=N(Comment)\r\nd,e,f', + config: { comments: '=N(' }, + description: "Comment, but bad char specified (comments: \"=N(\")", + notes: "Should silently disable comment parsing", + expected: { + data: [['a', 'b', 'c'], ['=N(Comment)'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: '#commented line\r\n', + config: { comments: true, delimiter: ',' }, + description: "Input with only a commented line (comments: true)", + expected: { + data: [], + errors: [] + } + }, + { + input: '#commented line', + config: { delimiter: ',' }, + description: "Input with comment without comments enabled", + expected: { + data: [['#commented line']], + errors: [] + } + }, + { + input: 'a\r\n b\r\nc', + config: { delimiter: ',' }, + description: "Input without comments with line starting with whitespace", + notes: "\" \" == false, but \" \" !== false, so === comparison is required", + expected: { + data: [['a'], [' b'], ['c']], + errors: [] + } + }, + { + input: 'a#b#c\r\n# Comment', + config: { delimiter: '#', comments: '#' }, + description: "Comment char same as delimiter", + notes: "Comment parsing should automatically be silently disabled in this case", + expected: { + data: [['a', 'b', 'c'], ['', ' Comment']], + errors: [] + } + }, + { + input: '\r\na,b,c\r\nd,e,f', + description: "Blank line at beginning", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\n\r\nd,e,f', + description: "Blank line in middle", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\nd,e,f\r\n\r\n', + description: "Blank lines at end", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\n \r\nd,e,f', + description: "Blank line in middle with whitespace", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\n,e,f', + description: "First field of a line is empty", + expected: { + data: [['a', 'b', 'c'], ['', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\n,e,f', + description: "First field of a line is empty", + expected: { + data: [['a', 'b', 'c'], ['', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,,c\r\n,,', + description: "Other fields are empty", + expected: { + data: [['a', '', 'c'], ['', '', '']], + errors: [] + } + }, + { + input: '', + description: "Empty input string", + expected: { + data: [], + errors: [{ + "type": "Delimiter", + "code": "UndetectableDelimiter", + "message": "Unable to auto-detect delimiting character; defaulted to comma" + }] + } + }, + { + input: ',', + description: "Input is just the delimiter (2 empty fields)", + expected: { + data: [['', '']], + errors: [] + } + }, + { + input: 'Abc def', + description: "Input is just a string (a single field)", + expected: { + data: [['Abc def']], + errors: [ + { + "type": "Delimiter", + "code": "UndetectableDelimiter", + "message": "Unable to auto-detect delimiting character; defaulted to comma" + } + ] + } + }, + { + input: 'a,b,c\r\nd,e,f\r\ng,h,i', + config: { preview: 0 }, + description: "Preview 0 rows should default to parsing all", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']], + errors: [] + } + }, + { + input: 'a,b,c\r\nd,e,f\r\ng,h,i', + config: { preview: 1 }, + description: "Preview 1 row", + expected: { + data: [['a', 'b', 'c']], + errors: [] + } + }, + { + input: 'a,b,c\r\nd,e,f\r\ng,h,i', + config: { preview: 2 }, + description: "Preview 2 rows", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f']], + errors: [] + } + }, + { + input: 'a,b,c\r\nd,e,f\r\ng,h,i', + config: { preview: 3 }, + description: "Preview all (3) rows", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']], + errors: [] + } + }, + { + input: 'a,b,c\r\nd,e,f\r\ng,h,i', + config: { preview: 4 }, + description: "Preview more rows than input has", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']], + errors: [] + } + }, + { + input: 'a,b,c\r\nd,e,"f\r\nf",g,h,i', + config: { preview: 2 }, + description: "Preview should count rows, not lines", + expected: { + data: [['a', 'b', 'c'], ['d', 'e', 'f\r\nf', 'g', 'h', 'i']], + errors: [] + } + } +]; \ No newline at end of file diff --git a/tests/test-runner.js b/tests/test-runner.js new file mode 100644 index 0000000..ec4bb96 --- /dev/null +++ b/tests/test-runner.js @@ -0,0 +1,171 @@ +var passCount = 0; +var failCount = 0; + +$(function() +{ + // First, wireup! + $('#results').on('click', 'td.rvl', function() + { + var tr = $(this).closest('tr'); + if (tr.hasClass('collapsed')) + { + $('.revealer', tr).hide(); + $('.hidden', tr).show(); + $(this).html("-"); + } + else + { + $('.revealer', tr).show(); + $('.hidden', tr).hide(); + $(this).html("+"); + } + tr.toggleClass('collapsed expanded'); + }); + + $('#expand-all').click(function() + { + $('.collapsed .rvl').click(); + }); + + $('#collapse-all').click(function() + { + $('.expanded .rvl').click(); + }); + + + + // Next, run tests and render results! + for (var i = 0; i < TESTS.length; i++) + { + var test = TESTS[i]; + var passed = runTest(test, i); + if (passed) + passCount++; + else + failCount++; + } + + + + // Finally, show the overall status. + if (failCount == 0) + $('#status').addClass('status-pass').html("All "+passCount+" test"+(passCount == 1 ? "" : "s")+" passed"); + else + $('#status').addClass('status-fail').html(""+failCount+" test"+(failCount == 1 ? "" : "s")+" failed; "+passCount+" passed"); +}); + +function runTest(test, num) +{ + var actual = Papa.parse(test.input, test.config); + + var results = compare(actual.data, actual.errors, test.expected); + + var testDescription = (test.description || ""); + if (testDescription.length > 0) + testDescription += '
'; + if (test.notes) + testDescription += '' + test.notes + ''; + + var tr = '' + + '+' + + '' + testDescription + '' + + passOrFailTd(results.data) + + passOrFailTd(results.errors) + + '
condensed
' + + '
condensed
' + + '
condensed
' + + '
condensed
' + + ''; + + $('#results').append(tr); + + if (!results.data.passed || !results.errors.passed) + $('#test-'+num+' td.rvl').click(); + + return results.data.passed && results.errors.passed +} + +function compare(actualData, actualErrors, expected) +{ + var data = compareData(actualData, expected.data); + var errors = compareErrors(actualErrors, expected.errors); + return { + data: data, + errors: errors + } +} + +function compareData(actual, expected) +{ + var passed = true; + + if (actual.length != expected.length) + passed = false; + + for (var row = 0; row < expected.length; row++) + { + if (actual.length != expected.length) + { + passed = false; + break; + } + + for (var col = 0; col < expected[row].length; col++) + { + if (actual[row].length != expected[row].length) + { + passed = false; + break; + } + + var expectedVal = expected[row][col]; + var actualVal = actual[row][col]; + + if (actualVal !== expectedVal) + { + passed = false; + break; + } + } + } + + // We pass back an object right now, even though it only contains + // one value, because we might add details to the test results later + // (same with compareErrors below) + return { + passed: passed + }; +} + +function compareErrors(actual, expected) +{ + var passed = JSON.stringify(actual) == JSON.stringify(expected); + + return { + passed: passed + }; +} + +function passOrFailTd(result) +{ + if (result.passed) + return 'OK'; + else + return 'FAIL'; +} + +function revealChars(txt) +{ + // Make spaces and tabs more obvious when glancing + txt = txt.replace(/( |\t)/ig, '$1'); + + txt = txt.replace(/(\r\n|\n\r|\r|\n)/ig, '$1$1'); + + // Now make the line breaks within the spans actually appear on the page + txt = txt.replace(/">\r\n<\/span>/ig, '">\\r\\n'); + txt = txt.replace(/">\n\r<\/span>/ig, '">\\n\\r'); + txt = txt.replace(/">\r<\/span>/ig, '">\\r'); + txt = txt.replace(/">\n<\/span>/ig, '">\\n'); + + return txt; +} \ No newline at end of file diff --git a/tests/tests.css b/tests/tests.css new file mode 100644 index 0000000..9fba093 --- /dev/null +++ b/tests/tests.css @@ -0,0 +1,159 @@ +/* Eric Meyer's Reset CSS v2.0 */ +html,body,div,span,applet,object,iframe,h1,h2,h3,h4,h5,h6,p,blockquote,pre,a,abbr,acronym,address,big,cite,code,del,dfn,em,img,ins,kbd,q,s,samp,small,strike,strong,sub,sup,tt,var,b,u,i,center,dl,dt,dd,ol,ul,li,fieldset,form,label,legend,table,caption,tbody,tfoot,thead,tr,th,td,article,aside,canvas,details,embed,figure,figcaption,footer,header,hgroup,menu,nav,output,ruby,section,summary,time,mark,audio,video{border:0;font-size:100%;font:inherit;vertical-align:baseline;margin:0;padding:0}article,aside,details,figcaption,figure,footer,header,hgroup,menu,nav,section{display:block}body{line-height:1}ol,ul{list-style:none}blockquote,q{quotes:none}blockquote:before,blockquote:after,q:before,q:after{content:none}table{border-collapse:collapse;border-spacing:0} + +body { + background: #F0F0F0; + font: 14px 'Helvetica Neue', sans-serif; + color: #333; + padding: 30px 15px; +} + +a { + color: rgb(0, 142, 236); +} + +a:hover { + color: rgb(82, 186, 255); +} + +b { + font-weight: bold; +} + +i { + font-style: italic; +} + +h1 { + text-align: center; + font-weight: bold; + font-size: 62px; + margin-bottom: 30px; +} + +.status-pass, +.status-fail { + padding: 10px; + margin-bottom: 30px; + color: #FFF; + text-align: center; + text-transform: uppercase; + font-size: 18px; + letter-spacing: 1px; + font-weight: 100; +} + +.status-pass { + background: rgb(3, 168, 3); +} + +.status-fail { + background: #BB0000; +} + +table { + width: 100%; + border-collapse: collapse; + margin-top: 5px; +} + +table th, +table td { + padding: 5px; + border: 1px solid #BBB; +} + +table th { + color: #000; + background: #DDD; + font-weight: bold; + padding: 10px 5px; + text-transform: uppercase; +} + +table td { + background: #FFF; + color: #555; + font-size: 14px; +} + +td.ok, +td.fail { + text-transform: uppercase; + font-weight: 300; + vertical-align: middle; + text-align: center; + width: 80px; +} + +td.ok { + background: rgb(204, 250, 144); +} + +td.fail { + background: rgb(255, 192, 192); +} + +td.rvl { + background: #444; + color: #999; + vertical-align: middle; + text-align: center; + cursor: pointer; + width: 20px; +} + +td.rvl:hover { + color: #FFF; +} + +tr.collapsed td.revealable { + background: #ECECEC; + vertical-align: middle; + text-align: center; + font-family: 'Helvetica Neue', sans-serif; + text-transform: lowercase; + color: #AAA; +} + +tr.expanded .revealer { + font-family: 'Helvetica Neue', sans-serif; + text-transform: lowercase; + font-size: 10px; + background: #FFF; + position: absolute; + display: block; + padding: 3px; + top: -5px; + right: -5px; +} + +td .container { + position: relative; +} + +.notes { + color: #888; + font-size: 12px; +} + +.pre { + font-family: Menlo, Monaco, monospace; + white-space: pre-wrap; +} + +td.pre { + font-size: 12px; +} + +.hidden { + display: none; +} + +.special-char { + color: #78B7E7; +} + +.whitespace-char { + background: #D5FCFA; +} \ No newline at end of file diff --git a/tests/tests.html b/tests/tests.html new file mode 100644 index 0000000..ed3f304 --- /dev/null +++ b/tests/tests.html @@ -0,0 +1,34 @@ + + + + Papa Parse Tests + + + + + + + + +

Papa Parse Tests

+ +
+ + Expand all + · + Collapse all +
+ + + + + + + + + + + +
Test CaseDataErrorsConfigInputExpectedActual
+ + \ No newline at end of file