Commit Graph

598 Commits

Author SHA1 Message Date
Alexander
3faf9450da
Merge pull request #40 from lafriks-fork/feat/strategy_xml
Add XML strategy
2020-12-02 00:10:52 +01:00
Lauris BH
0596fda1a4
Fix strategy order 2020-11-26 13:56:25 +02:00
Alexander
6edbff3dec
Merge pull request #38 from softagram/fix-readme-cmd
Fix typo in the pip command in README.md
2020-11-26 12:46:28 +01:00
Alexander
8de21f365e
Merge pull request #39 from lafriks-fork/feat/sync_7.12.1
Sync with linguist 7.12.1
2020-11-26 12:38:46 +01:00
Lauris BH
8ac98f4b77 Update readme 2020-11-15 15:48:03 +02:00
Lauris BH
6d8f15af5b Add XML strategy 2020-11-15 15:43:37 +02:00
Lauris BH
289ac3d9f0 Sync with linguist 7.12.1 2020-11-15 14:32:56 +02:00
Ville Laitila
8d83871580 Fix typo in the pip command in README.md 2020-11-14 23:29:53 +02:00
Alexander
0fb4b8a768
Merge pull request #35 from lafriks-fork/feat/manpage_strategy
Add support for Roff man pages filenames
2020-10-22 00:10:39 +02:00
Alexander
7688057adc
Merge pull request #37 from lafriks-fork/sync_7_11_1
sync to the latest github/linguist v7.11.1
2020-10-22 00:08:04 +02:00
Lauris BH
bc76dd38b0 sync to the latest github/linguist v7.11.1 2020-10-12 12:32:48 +03:00
Lauris BH
cb353b4b05 Add support for Roff man pages filenames 2020-10-12 12:18:57 +03:00
Alexander
d7f6b27b7d
Merge pull request #34 from lafriks-fork/sync_7_11
sync to the latest github/linguist v7.11.0
2020-09-24 12:24:42 +02:00
Lauris BH
7c562a6c34 sync to the latest github/linguist v7.11.0 2020-09-17 10:34:41 +03:00
Alexander
5717abd4c0
Merge pull request #30 from bzz/python-ci
CI for Python bindings
2020-08-17 11:56:30 +02:00
Alexander Bezzubov
e98983b3f9 ci: add Python tests profile (\wo gopy)
Signed-off-by: Alexander Bezzubov <alexander.bezzubov@jetbrains.com>
2020-08-12 15:23:01 +02:00
Alexander Bezzubov
328c16f948 py: use readme as pypy description
Signed-off-by: Alexander Bezzubov <alexander.bezzubov@jetbrains.com>
2020-08-12 15:22:55 +02:00
Alexander Bezzubov
7ee65cc9d0 doc: upd build instructions
Signed-off-by: Alexander Bezzubov <alexander.bezzubov@jetbrains.com>
2020-08-12 15:22:50 +02:00
Alexander
5d58b1aaaf
Merge pull request #29 from vsmaxim/master
python: cover the rest of python bindings from shared library, add tests, add docstrings for API
2020-08-12 14:35:58 +02:00
Maxim Vasilev
59f0f17834 Remove unneded todos 2020-08-11 00:29:33 +03:00
Maxim Vasilev
08bc9bca0e Cover the rest of python bindings from shared library, add tests, add docstrings, add setup.py. 2020-08-11 00:12:43 +03:00
Máximo Cuadros
dc6fc02209
Merge pull request #24 from erizocosmico/fix/bail-out-if-not-enough-lines
data: bailout in some cases if there arent enough lines
2020-05-28 16:45:10 +02:00
Miguel Molina
78696c2272
data: bailout in some cases if there arent enough lines
Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
2020-05-28 13:39:59 +02:00
Máximo Cuadros
2880ccae4a
Merge pull request #23 from erizocosmico/fix/get-first-line
data: fix getting the first line for empty content
2020-05-28 11:52:49 +02:00
Miguel Molina
79398a925d
data: fix getting the first line for empty content
Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
2020-05-28 11:28:49 +02:00
Máximo Cuadros
e1f1b57a84
Merge pull request #22 from erizocosmico/feature/generated
implement IsGenerated helper to filter out generated files
2020-05-28 10:34:37 +02:00
Miguel Molina
8ff885a3a8
implement IsGenerated helper to filter out generated files
Closes #17

Implements the IsGenerated helper function to filter out generated
files using the rules and matchers in:
- https://github.com/github/linguist/blob/master/lib/linguist/generated.rb

Since the vast majority of matchers have very different logic, it cannot
be autogenerated directly from linguist like other logics in enry, so it's
translated by hand.

There are three different types of matchers in this implementation:
- By extension, which mark as generated based only in the extension. These
  are the fastest matchers, so they're done first.
- By file name, which matches patterns against the filename. These
  are performed in second place. Unlike linguist, we try to use string
  functions instead of regexps as much as possible.
- Finally, the rest of the matchers, which go into the content and try
  to identify if they're generated or not based on the content. Unlike
  linguist, we try to only read the content we need and not split it
  all unless it's necessary and use byte functions instead of regexps
  as much as possible.

Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
2020-05-28 08:55:13 +02:00
Máximo Cuadros
bda45fdc8e
go.mod: update go-oniguruma v1.2.1 2020-05-06 21:42:07 +02:00
Alexander
4b468762b6
Merge pull request #13 from go-enry/python-wrapper
Python: API to expose highest-level enry.GetLanguage
2020-04-24 20:57:37 +02:00
Alexander Bezzubov
35575d0a3e
py: expose highest-level enry.language()
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-04-24 20:51:46 +02:00
Máximo Cuadros
1d23012ae6
Merge pull request #15 from go-enry/ci-libonig5
ci: update libonig5 version
2020-04-24 19:24:34 +02:00
Máximo Cuadros
aa46bf4a37
ci: update libonig5 version 2020-04-24 19:18:27 +02:00
Máximo Cuadros
9fba3da45f
Merge pull request #12 from mcuadros/oniguruma
data: replace substring package with regex package
2020-04-15 19:58:28 +02:00
Máximo Cuadros
29bc0a181b
data: replace substring package with regex package 2020-04-15 17:27:48 +02:00
Máximo Cuadros
b34576bd71
Merge pull request #10 from mcuadros/is-test
IsTest function for top 10 languages
2020-04-06 16:29:11 +02:00
Máximo Cuadros
b851ee83ad
IsTest function for top 10 languages 2020-04-06 16:23:48 +02:00
Alexander Bezzubov
4fb0b4cc5e
doc: add coloring to the ToC
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-31 11:20:06 +02:00
Alexander
6c06680bef
Merge pull request #2 from lafriks-fork/feat/lang_groups
Return group color if language has none
2020-03-31 11:17:52 +02:00
Lauris BH
97a26011a9 Return group color if language has none 2020-03-31 09:30:27 +03:00
Alexander
cfaa7a1711
Merge pull request #8 from go-enry/update-docs
Documentation update
2020-03-30 22:53:10 +02:00
Alexander Bezzubov
6a09a2a684
doc: update badges
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-30 19:42:03 +02:00
Alexander Bezzubov
64d02e5441
doc: re-structure README by use case, update links
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-30 19:31:53 +02:00
Alexander
c0806d20c8
Merge pull request #3 from lafriks-fork/feat/sync_linguist_7_9_0
sync to the latest github/linguist v7.9.0
2020-03-30 12:28:03 +02:00
Lauris BH
9030d3671b sync to the latest github/linguist v7.9.0 2020-03-30 01:25:57 +03:00
Alexander
fa1c6f39b5
Merge pull request #7 from go-enry/win-support
Code generator Win support
2020-03-29 23:31:10 +02:00
Alexander Bezzubov
172486906a
ci: force git to use LF on win to pass tests on linguist samples
This mitigates the problem that tokenizer uses regex
that matches platform-specific line endings

TestPlan:
 - go test ./internal/code-generator/generator \
	-run Test_GeneratorTestSuite -testify.m TestTokenizerOnATS

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-29 23:23:56 +02:00
Alexander Bezzubov
3ea961e5ab
generator: change-detector tests on EOL-dependant sample
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-29 23:23:56 +02:00
Alexander Bezzubov
9be0211f04
generator: skip symlinks on *nix and win
As Git on win does not support symlinks [1], we have to hard-code
the paths to fils under ./samples/ in Linguist codebase that are
known to be a symlink.

 1. https://github.com/git-for-windows/git/wiki/Symbolic-Links

TestPlan:
 - go test ./internal/code-generator/generator -run Test_GeneratorTestSuite

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-29 23:23:56 +02:00
Alexander Bezzubov
9c082eb2d4
ci: add ENRY_DEBUG flag
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-29 19:35:49 +02:00
Alexander Bezzubov
78eee0cf7e
generator: flag to debug building of bayesian classifier
It seems that reading ./samples/ from Linguist consumes
a different number of files from filesystem on different OSes.

This change adds ENRY_DEBUG env var to print some debug output
about calculations of token stats from  samples.

TestPlan:
 - ENRY_DEBUG=1 go test -v ./internal/code-generator/generator \
	-run Test_GeneratorTestSuite -testify.m TestGenerationFiles

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2020-03-29 19:35:49 +02:00