Identifier coloring and whitespace

Jungle Ide Home Forums Historic logs New features log Identifier coloring and whitespace

This topic contains 29 replies, has 2 voices, and was last updated by  Ziggy 6 years, 10 months ago.

Viewing 25 posts - 1 through 25 (of 30 total)
  • Author
    Posts
  • #2605

    Goodlookinguy
    Participant

    I wasn’t sure if I should put this as a bug at first, but then I figured that it definitely was non-consistent behavior on the coloring part.

    I think the identifiers should to be broken down into separate groups. It gets very strange looking if I have a field, like “Map”, or a method, like “Set”, that’s one color but then my other fields or methods are another color.

    The picture I attached is an example of this non-consistent coloring issue and should not be run.
    ze356-SetColor.png

    EDIT: Parentheses, plus signs, minus signs, and every other symbol is being colored by the whitespace setting. This is definitely a bug.

    #3382

    Ziggy
    Keymaster

    That’s not exactly an inconsitence on method colorization. This is caused becouse you’re importing the Set class, and the Jungle Ide parser is coloring the Set word because it’s the name of a Class.

    On Jungle Ide coloring of text is done on the tokenizing process, so when a word is colored, Jungle Ide does not have context information, so it cannot differenciate if it’s a Class or a Method name. Doing it the other way round, that is, colorizing text on the lexing phase, would have a very big performance impact, so it is more a inevitable side effect to get things being colored fast.
    “Set” is a built-in class defined in the “set.monkey” file in the monkey folder that contains the built-in monkey modules. That’s causing the confusion.

    #3383

    Goodlookinguy
    Participant

    I’m sorry, I don’t think I made myself very clear. I’m not confused at all. I know why it’s coloring everything the way it is. I’m calling the coloring inconsistent because you see the two methods and one is one color while the other is another. While I understand why it’s doing that, I also think that it’s possible to fix without too much of a performance drop. Visual Studio, Dreamweaver, Zend Studio, Eclipse, and many other IDEs have done it. So I imagine it’s quite possible.

    One way I could see doing this is to still color everything like it is now, but add an extra coloring routine during the lexing phase if it matches only certain patterns.

    EDIT: Since I know you do have to make this run for a variety of computers, I suppose an extra routine could be heavy for some. However, all of my computers handle Jungle IDE like it’s nothing. An optional extra routine that could fix this problem would be nice for those of us who have computers powerful enough to deal with an IDE in its entirety.

    #3384

    Ziggy
    Keymaster

    I’ll be looking to this. I do not promis anything soon until I get a real idea of how this can interfere on a very large project, but I’ll provide feedback of my test and if it si not as heavy as I thought, I’ll implement it.

    #3388

    Ziggy
    Keymaster

    I’ve been making some test. I’ve added an additional parsing phase on the syntax renderer that allows post-lexing coloring but it does have a terrible performance hit on current design. I *think* I’ll be able to fix this by pre-catching token-kind recognition so all this is only re-calculated on text changes. Not sure yet it’ll be easy to integrate without loosing performance, but I think it’s possible to have something usabe. I’ll keep on investigating. The good neews is that, if this works, we will be able to have specific coloring for methods, classes, consts, locals, etc. all of them diferenciated properly and scope-based. But I don’t want to say this is going to happen soon, it’s not exactly a simple addition.

    #3390

    Goodlookinguy
    Participant

    Very nice, keep it up. I’ll be very interested to see what happens.

    #3391

    Ziggy
    Keymaster

    I’ve managed to integrate a cached version of it, and it is a bit faster but you can notice the performance hit on a middle-size solution in my quad-core computer so it’s a bit overkill. anyway, I really like how it looks and the fact that coloring is way more exact this way so I think I’ll leave it as an option in the preferences dialog, if I find it stable. I need to change also some internal parts of the current lexer and I’m not sure if this will have any side effects. I’ll keep you informed.
    Are you open to beta-testing this new lexer in your own projects?

    #3392

    Ziggy
    Keymaster

    I think I have something usable, but needs some testing. Please, if you want to give it a go before I release to everybody, it’ll be great.
    I’m moving this to feature requests as I think it is more of a new feature to the Monkey parser than a bug on current implementation.

    #3393

    Ziggy
    Keymaster

    I think you’ll like this picture. it’s prety self-explanatory of what I think I will be able to provide very soon on Jungle:
    1zv5a-NewParser.png

    #3394

    Goodlookinguy
    Participant

    WOW! Surprised me how quickly you got that working. Great job! I’m definitely open to beta testing it with my stuff.

    #3395

    Ziggy
    Keymaster

    Great! I’ve sent a build to the email in your jungleide.com profile. It’s a installer, just run it and it’ll auto-install replace current version. (you can disable the new parser on the preferences dialog if it breaks anything)

    #3397

    Goodlookinguy
    Participant

    Alright, so some problems I noticed right away. Not with the speed. It runs pretty well and every so often is a light flicker, which I assume to be the split second between the coloring phases.

    The issues I found was not the speed, but a new coloring issue. I purposefully wrote this page as a test, it shouldn’t actually run.

    In the first two images, notice that there’s a method and function called Set. Just those being there will change the colors of every place the word “Set” is. Whichever one of those is declared first is the color of the word set from that point on.

    Set function before method
    jo35k-Set-Func-before-Method.png

    Set function after method
    7a2z0-Set-Func-after-Method.png

    parameters are not colored
    67vu8-Parameter-Coloring.png

    EDIT: OH! I completely forgot that generic types and primitive shortcuts are not colored. Not that I use shortcuts, but looking at map was interesting.

    Parameter%20Coloring%20and%20Shortcut%20Primitives.png

    EDIT: Another thing, though I’m not sure if it’s an issue, but it seems like one. The option “Advanced parser Class name” appears 3 times.
    Advanced%20Parser%20Class%20Name.png

    #3398

    Ziggy
    Keymaster

    Hi, to the function before/after method. That’s expected. Jungle does not still diferenciate methods with the same name as functions (I am surprised this is allowed in Monkey as long as they have different parameters signature). I’ll be adding this at some point, but not soon, as this would complicate even a bit more the parsing, and I think we need to be sure everything else is stable first. So, by now, avoid having methods and functions with the same name in the same class. (I would considere this also a good coding practicle).
    To the parameters not being properly colores, I’ll be fixing this soon.
    And the primitive datatypes shortcuts, this is a bit more complicated to solve as it affects the tokenizer, and it can have all kind of side effects, but I’ll be looking at this soon.
    The slight flickering every once in a while, it’s expected as there are now two parsers running. I will be reducing it in the future as long as I optimize a bit everything.

    Edit: The Class Name advanced parser is a bug. I’ve just fixed it.

    #3400

    Ziggy
    Keymaster

    Ok, other than the function/method overloads with same name in the same class, everything seems to beworking great now. Almost ready for an official release :D

    #3401

    Goodlookinguy
    Participant

    In the first picture I’m attaching, only functions have underscores. The class “Set” shouldn’t be that color and have underscores. In the second picture, whitespace has underscores. For some reason all of the symbols are considered whitespace.
    0kp7h-Method-Function-Coloring.png
    2gp2q-Whitespace-and-Bleeding-Colors.png

    Edit: More examples of color bleeding and whitespace issue. I put underscores on functions for this picture, so that’s not a issue. (Taken straight from mojo/graphics.monkey)
    More%20Color%20Bleeding.png

    Keep up the great work. It really isn’t that far off from a final release. With the speed that you made all of this work, I imagine you’ll fix these in no time.

    #3403

    Ziggy
    Keymaster

    Ok, I won’t be fixing the problems coloring the underscore on whitespaces yet (I will but not inmediatly). “whitespaces” is the name of the text style used for most undefined token separators. I supose I should have choosen a different name for this. The fact that separators without a graphical representation such as white space and tabs are highlighted using a different color sometimes is due to them being mixed with a neighbour token, as a side effect of some rendereing optimizations. I think the benefits are greater than the drawbacks, so I’ll leave this particular visualization issue for a future fix.

    #3404

    Goodlookinguy
    Participant

    Other than the first image, there’s really nothing that looks wrong in my code. I was working in it for the last two hours and it’s been super fast and the flickering seems to have been reduced significantly from the first version I tried. I don’t know if you changed something or if I shut down some programs that might have been slowing it down mildly, but you’ve done a great job.

    #3406

    Ziggy
    Keymaster

    Yes, the first image is caused becouse, in your sample, the best candidate to be solved for the word “Set” is the function inside the class, as it is the item with this name in the closest nesting level BUT, what Jungle is doing wrong, is that on the speciffic context of class declaration or instantation, there’s no chance to refer to a function, so as there are no clashes, it should bypass the first “match” and continue solving the reference on an upper nesting level, until it finds the reference in the monkey/set.monkey module. It’s hard to let jungle knows when it has to look for a regular identifier, and when it has to limit its refactoring to classes, interfaces and modules. I’ll be fixing this soon, as the following operators can help solving this: “:” “New” “Extends” I’ll have to add a sub-tokenizer that evaluates expression in order to also identify classes inside the generic declaration. As Mark insisted on using the < and > symbols for both comparsions and generics declarations, this will complicate things and make the parsing a bit slower… Not sure when I’ll have this sorted in a reasonably fast way, but hopefully it’ll be soon

    #3411

    Goodlookinguy
    Participant

    New bizarre bug. It colors a word partway through. It should be possible to replicate. What you have to do is select at least two lines of text and stop partway through a word and then click on a line that isn’t the one that the selection tool is halfway through highlighting.
    3s82q-Highlight-Bug.png

    #3412

    Ziggy
    Keymaster

    I supose I have to force an “invalidate” message on every “deselect” operation. Not sure how to handle this from within windows…

    #3413

    Ziggy
    Keymaster

    Ok, the selction visualization bug has been fixed. It was easier than I first thought.

    #3419

    Goodlookinguy
    Participant

    Some fun new issues I found. Now there’s an issue with New parameter coloring and if you make a new class that the parameters lose their coloring. The other one is the parameter of the size of a new array, doesn’t get colored.

    e1do7-New-Parameter-Color.png
    92g5s-New-Int-Size.png

    Edit: Problem number two is a little bit bigger than I thought…
    Bracket%20Double%20Dot.png

    #3420

    Ziggy
    Keymaster

    Thanks! Double dot was confusing the parser. I’ve already solved. The arrays issue will be longer to solve… arrrgssss

    EDIT: I have arrays aparently working. I’ll keep on testing…

    #3421

    Goodlookinguy
    Participant

    So the New parameters are kind-of colored now.
    http://www.jungleide.com/wp-content/uploads/2012/05/x93a0-Obscure-New-Parameter-Coloring.png

    Nevermind, for some reason I put the wrong parameter variable name :| …

    Well, aside from the shortcut operator stuff and the New keyword turning red if you have a constructor inside a class, I’d say it’s ready. It’s running very fast over here.

    This picture shows an overview of the last things I tested and the New keyword being colored red when within a class with a constructor. After opening up numerous tests I had done, everything seems to be good to go.
    Good%20To%20Go.png

    #3422

    Ziggy
    Keymaster

    What’s the “shortcut operatot stuff”?

Viewing 25 posts - 1 through 25 (of 30 total)

The topic ‘Identifier coloring and whitespace’ is closed to new replies.