Commit Graph

384 Commits

Author SHA1 Message Date
Alexander Bezzubov
8756fbdcb4
refactor to build tags
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-14 21:38:16 +02:00
Alexander Bezzubov
553399ed76
tokenizer: port flex-based C impl from linguist
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-14 21:38:16 +02:00
Alexander
ab3c26b46d
Merge pull request #224 from bzz/ci-fix-linux-shared-release
HOTFIX: enable cgo for linux native lib release on CI
2019-04-12 00:14:58 +02:00
Alexander Bezzubov
ade47c0a22
ci: enable cgo for linux native lib release
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 23:25:27 +02:00
Alexander
8c5e0ceaca
Merge pull request #222 from bzz/ci-workaround
CI: workaround for incompatible compiler release binary
2019-04-11 23:02:24 +02:00
Alexander Bezzubov
85d5906b2b
address review feedback - tixing a fypo
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 21:36:29 +02:00
Alexander Bezzubov
85de120c17
ci: export env vars
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 21:35:49 +02:00
Alexander Bezzubov
41478262f3
fix verb mismatch in a format string
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 15:28:49 +02:00
Alexander Bezzubov
110822b014
ci: remove 'sudo: false' as it's deprecated on Travis
See https://docs.travis-ci.com/user/reference/trusty/#container-based-infrastructure
and https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration#timeline---its-happening-fast

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 15:10:56 +02:00
Alexander Bezzubov
5fab94b5b9
ci: disable cgo by default
With go1.11 `go test` in GOPATH mode somehow
seems to depend on GCC. See https://github.com/golang/go/issues/28065

This change only enables cgo for CI profiles that
need it. Those are the ones that seem to fail
on TravisCI now, presumably due to some compiler
version missmatch.

That is a workaround and does not happen in GO11MODULE mode.

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 15:10:43 +02:00
Alexander Bezzubov
58a81694f6
ci: fix bug that enabled oniguruma for all profiles
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 15:09:47 +02:00
Alexander Bezzubov
d73725709b
ci: fix versions to workaround compiler release 🐛
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-11 15:08:32 +02:00
Alexander
59289bd4f5
Merge pull request #220 from bzz/skip-jar-upload
Skip uploading useless jar as part of Github release
2019-04-09 18:39:55 +02:00
Alexander Bezzubov
82f784d2f7
release: skip uploading useless jar; fixes #208
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-09 18:27:07 +02:00
Alexander
7a6e8ca783
Merge pull request #214 from bzz/fix-cli-accuracy
CLI: sync report logic \w Linguist
2019-04-09 16:32:52 +02:00
M. J. Fromberger
bad147cb72
Update README.md
Co-Authored-By: bzz <bzz@users.noreply.github.com>
2019-04-08 18:00:43 +02:00
Alexander Bezzubov
bdb5603f28
Address code review feedback
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-08 16:07:10 +02:00
Alexander Bezzubov
416afb45fc
doc: better wording in 'divergences from linguist' section
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-08 15:58:46 +02:00
Alexander
db6073efa6
Merge pull request #217 from bzz/single-file-add-vendor
CLI: always report vendoring for a single file
2019-04-08 11:41:05 +02:00
Alexander Bezzubov
b6027d6d0c
cli: mode=byte by default + fix file reading
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-04 22:02:30 +02:00
Alexander Bezzubov
ace8af223f
cli: report vendoring for single file
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-04 17:04:16 +02:00
Alexander
33cc83f473
Merge pull request #211 from SuhaibMujahid/patch-1
Update README.md
2019-04-04 15:32:13 +02:00
Alexander Bezzubov
94e8598d3d
doc: update TravisCI links
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-04 15:27:12 +02:00
Alexander
32336ac554 Update README.md
Co-Authored-By: SuhaibMujahid <suhaibmujahid@gmail.com>
Signed-off-by: Suhaib Mujahid <suhaibmujahid@gmail.com>
2019-04-03 14:59:31 -04:00
Alexander Bezzubov
c9f1793a78
doc: update godoc and README \w supported features
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-03 18:51:54 +02:00
Alexander Bezzubov
88810fed12
cli: mimic linguist output by default
This includes next main changes:

 - default: print only Programming and Markup types
   as Linguist does
 - `-prog` option replaced with `-all`, to allow for
   previous behavior
 - always use GetLanguage as main source of truth
   that fixes #204 and perf will be restored under #212

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-03 16:21:10 +02:00
Alexander Bezzubov
df01124e18
doc: better wording in API godoc
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-03 16:07:14 +02:00
Alexander Bezzubov
b2b61c2a8c
gen: refactoring, renaming vars for readability
This does not change the logic of the generatro
but only renames/moves some vars for readability

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-04-03 15:40:23 +02:00
Suhaib Mujahid
badbfd06e5 Update README.md
Signed-off-by: Suhaib Mujahid <suhaibmujahid@gmail.com>
2019-04-01 18:49:01 -04:00
Alexander
6526da77b3
Merge pull request #205 from kuba--/fix-langs
Do not return empty lang.
2019-03-15 18:51:55 +01:00
Alexander
56214e3289
Merge pull request #206 from kuba--/go-mod
Replace archived rubex by src-d/go-oniguruma
2019-03-15 16:19:38 +01:00
kuba--
14281d1785
Replace archived rubex by src-d/go-oniguruma
Signed-off-by: kuba-- <kuba@sourced.tech>
2019-03-14 16:09:25 +01:00
kuba--
5adfee5761
Do not return empty lang.
It's better to return any potential candidate than nothing.

Signed-off-by: kuba-- <kuba@sourced.tech>
2019-03-14 14:08:19 +01:00
Alexander
e067e45044
Merge pull request #202 from bzz/go-11
ci: add go 1.11
2019-02-21 11:49:23 +01:00
M. J. Fromberger
3a6d42b39a
doc: fix spelling
Co-Authored-By: bzz <bzz@users.noreply.github.com>
2019-02-21 09:33:17 +01:00
Alexander Bezzubov
baefa18475
gen: compare generated code to gold ignoring whitespaces
Reason is that gofmt can change between versions e.g
see https://go-review.googlesource.com/c/go/+/122295/
and this would avoid breaking tests and edit wars

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-02-20 23:22:02 +01:00
Alexander Bezzubov
c8e0f75132
test: make gen test output less verbose
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-02-20 23:22:02 +01:00
Alexander Bezzubov
01633d410f
ci: make java release use latest go code
Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-02-20 23:22:01 +01:00
Alexander Bezzubov
f42b68fe70
ci: multiple go versions in build matrix
Mostly affects enry-java release, now using 1.11

Signed-off-by: Alexander Bezzubov <bzz@apache.org>
2019-02-20 23:22:01 +01:00
Alexander
3499750785
Sync to linguist 7.2.0: heuristics.yml support (#189)
Sync \w Github Linguist v7.2.0

Includes new way of handling `heuristics.yml` and
all `./data/*` re-generated using Github Linguist [v7.2.0](https://github.com/github/linguist/releases/tag/v7.2.0)
release tag.

 - many new languages
 - better vendoring detection
 - update doc on update&known issues.
2019-02-14 12:47:45 +01:00
Alexander
13d3d66d37
refactoring: remove un-used code, add go doc, fix ci (#199)
Refactoring, consisting of
 - remove unused method `isAuxiliaryLanguage` and `FileCountList`
   in order to reduce public API surfaces (go/java)
 - add GoDoc to public APIs
 - ci: java profile use latest go src
  It also now mimics https://docs.travis-ci.com/user/languages/go/#go-import-path
  for non-go build image, as code relies on internal imports.

TEST PLAN:
 - make test
2019-02-05 22:54:14 +01:00
Alexander
fe18dc0830
Merge pull request #198 from creachadair/indocumentado
Add documentation comments to package tokenizer.
2019-01-29 21:18:56 +01:00
M. J. Fromberger
5245079744 Apply suggestions from review.
Signed-off-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2019-01-29 11:28:44 -08:00
M. J. Fromberger
dabb41527f Apply suggestions from review.
Signed-off-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2019-01-29 11:28:42 -08:00
M. J. Fromberger
4027b494b3 Add documentation comments to package tokenizer.
Although this package is internal, it still exports an API and deserves some
comments. Serves in partial satisfaction of #195.

Signed-off-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2019-01-29 11:18:52 -08:00
M. J. Fromberger
260dcfe002
Merge pull request #197 from creachadair/muckthebits
Prevent tokenization from modifying its input.
2019-01-29 11:18:06 -08:00
M. J. Fromberger
7d277b11de Copy the tokenizer input to avoid modifying the caller's copy.
Addresses #196. Several of the tokenizer's processing steps wind up editing the
source, and we don't want those changes to be observed by the caller, which may
use the source for other purposes afterward.

Signed-off-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2019-01-29 10:12:33 -08:00
M. J. Fromberger
169060e1cd Add a test that tokenization does not modify the input.
At present this test fails, since the tokenizer replaces text in shared slices
of the input. A subsequent commit will fix that.

Signed-off-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2019-01-29 10:03:09 -08:00
M. J. Fromberger
3748186e13
Merge pull request #187 from creachadair/cleanup
Remove an unnecessary helper function.
2019-01-23 06:59:00 -08:00
M. J. Fromberger
8fcadc60dc Remove an unnecessary helper function.
The writeStringLn function repeats what fmt.Fprintln already does.  Since this
package already imports fmt, removing it reduces mass.

Signed-off-by: M. J. Fromberger <michael.j.fromberger@gmail.com>
2019-01-08 08:46:36 -08:00