Commit Graph

29 Commits

Author SHA1 Message Date
Matt Dietz
d37c4fd551 Drops Python 2 support
REP-1030

In addition to some python 2 => 3 fixes, this change bumps the scikit-learn
version to latest. The previously pinned version of scikit-learn failed trying
to compile all necessary C modules under python 3.7+ due to included header files
that weren't compatible with C the API implemented in python 3.7+.

Simultaneously, with the restrictive compatibility supported by scikit-learn,
it seemed prudent to drop python 2 support altogether. Otherwise, we'd be stuck
with python 3.4 as the newest possible version we could support.

With this change, tests are currently passing under 3.9.2.

Lastly, imports the original training data. At some point, a new version
of the training data was committed to the repo but no classifier was
trained from it. Using a classifier trained from this new data resulted
in most of the tests failing.
2021-06-10 14:03:25 -05:00
Sergey Obukhov
8138ea9a60 fix text with Date: misclassified as quotations splitter 2019-01-18 16:49:39 +03:00
Sergey Obukhov
0e6d5f993c fix appointments in text 2017-10-23 16:32:42 -07:00
Ezra Pagel
221774c6f8 android_wrote regex was incorrectly iterating characters in 'wrote', resulting in greedy regex that
matched many strings with dashes
2017-08-21 12:47:06 -05:00
Hung Nguyen
b8e1894f3b add test case 2017-07-10 13:28:33 +07:00
Ethan Setnik
cca64d3ed1 add test case 2017-04-11 23:36:36 -04:00
Sergey Obukhov
0f5e72623b add android quotation pattern 2017-04-10 16:33:21 -07:00
smitcona
29f1d21be7 fixed expected markers and incorrect condensed header not matching regex 2017-02-06 15:03:22 +00:00
smitcona
34c5b526c3 Remove the whitespace before the line if the flag is set 2017-02-03 12:57:26 +00:00
smitcona
984c036b6e Set the marker back to 'm' rather than 't' if it matches the QUOT_PATTERN. Updated test case. 2017-02-01 18:28:19 +00:00
smitcona
a403ecb5c9 Adding two level indentation test 2017-02-01 18:09:35 +00:00
smitcona
a44713409c Added additional case for testing new functionality of split_emails() 2017-02-01 17:40:59 +00:00
smitcona
b5e3397b88 Updating test to account for --original message-- case 2016-11-22 20:00:31 +00:00
smitcona
adfed748ce split_emails function added, test added 2016-11-21 12:35:36 +00:00
Umair Khan
cefbcffd59 Make tests/text_quotations_test.py compatible with Python 3. 2016-07-13 14:45:26 +05:00
Umair Khan
da998ddb60 Run modernizer on the code. 2016-07-12 17:25:46 +05:00
Sergey Obukhov
02adf53ab9 fixes mailgun/talon#12 2016-03-04 13:14:50 -08:00
Sergey Obukhov
999e9c3725 fixes mailgun/talon#19 2016-02-19 17:53:52 -08:00
Adam Renberg
14e3a0d80b Add better support for Scandinavian languages
This is a port of https://github.com/tictail/claw/pull/6 by @simonflore.
2015-09-21 21:42:01 +02:00
Sergey Obukhov
385285e5de process first 1000 lines for long messages, support for German and Dutch 2015-09-11 06:17:14 -07:00
Oliver Song
7ea773e6a9 Fix iphone test 2015-07-02 21:49:09 -04:00
Scott MacVicar
8b1f87b1c0 Get this building and passing tests
Changes:
* add .DS_Store to .gitignore
* Decode base64 encoded emails for tests
* Pick a version of scikit since the pickled clasifiers are based on that
* Add missing numpy and scipy dependencies
2015-07-02 21:49:09 -04:00
Alex Riina
f16760c466 Remove flanker and replace PyML with scikit-learn
I never was actually able to successfully install PyML but the source-forge
distribution and lack of python3 support convinced me that scikit-learn would
be a fine substitute. Flanker was also difficult for me to install and seemed
only to be used in the tests, so I removed it as well to get into a position
where I could run the tests. As of this commit, only one is not passing
(test_standard_replies with android.eml) though I'm not familiar with the `email`
library yet.
2015-07-02 21:49:09 -04:00
Simon
072a440837 Test cases for new patterns 2015-04-15 13:55:17 +02:00
szymonsobczak
3c9ef4653f some more french fromats 2015-02-24 12:18:54 +01:00
szymonsobczak
b16060261a support some polish and french formats 2015-02-24 11:39:12 +01:00
Jeremy Schlatter
3768d7ba31 make a separate test function for each language 2014-12-30 14:41:20 -08:00
Jeremy Schlatter
613d1fc815 Add extra splitter expressions and tests for German and Danish.
Also some refactoring to make it a bit easier to add more languages.
2014-12-23 15:44:04 -08:00
Sergey Obukhov
170f11038b initial commit 2014-07-23 21:12:54 -07:00