Data Portal @ linkeddatafragments.org

Matches in ESWC 2020 for { ?s ?p ?o. }

• Paper.191_Review.1 hasContent "The paper presents a framework for evaluate graph embeddings on several tasks. This framework includes both machine learning tasks (e.g., classification, regression, clustering) and semantic tasks (e.g., entity relatedness, document similarity). The paper is well written and clear. The framework is well explained and the given examples help to understand the user cases. The choices of tasks and parameters are reasonably justified. However, while I understand the rationale given for not including link prediction, I think that one of the main advantage of these kinds of framework is to be able to perform comprehensive evaluations that include all popular tasks. Therefore, it may be useful to include also this task in the future. I could not find the Permanent URL for the resource at the beginning of the paper. The authors should include one in the camera ready. The related work is well done and fairly comprehensive. However, I would suggest to briefly mention also the work relevant to the evaluation of ontology/knowledge graphs via relevant tasks (basically the “application and usage impact” category of the survey “Zablith et al. Ontology evolution: a process-centric survey”, e.g., https://www.zora.uzh.ch/id/eprint/174974/1/00-iswc2019-pernischova-dc.pdf, http://oro.open.ac.uk/55536/1/ISWC2018_Research.pdf). It would benefit the paper to include more details about the scalability of the frameworks. How does it handle parallelization? How long did it take for the evaluation discussed in section 5? What kind of machine did you use? How would the size of the vectors affect the computational time for the various tasks? The evaluation is not particularly comprehensive, since it focuses only on two tasks (classification and regression). Why not showcase all the tasks supported by the system? Figure 3 is not very readable. Please, improve it or change it to a table. Finally, I would suggest choosing a name for the framework, in order to facilitate people that would like to refer to it. In conclusion, the framework presented in the paper appears to be quite a useful resource for the communities of Semantic Web and Machine Learning."".
• Paper.191_Review.2_Reviewer type RoleDuringEvent.
• Paper.191_Review.2_Reviewer label "Anonymous Reviewer for Paper 191".
• Paper.191_Review.2_Reviewer withRole ReviewerRole.
• Paper.191_Review.2_Reviewer withRole AnonymousReviewerRole.
• Paper.191_Review.2 type ReviewVersion.
• Paper.191_Review.2 issued "2001-02-03T08:16:00.000Z".
• Paper.191_Review.2 creator Paper.191_Review.2_Reviewer.
• Paper.191_Review.2 hasRating ReviewRating.2.
• Paper.191_Review.2 hasReviewerConfidence ReviewerConfidence.3.
• Paper.191_Review.2 reviews Paper.191.
• Paper.191_Review.2 issuedAt easychair.org.
• Paper.191_Review.2 issuedFor Conference.
• Paper.191_Review.2 releasedBy Conference.
• Paper.191_Review.2 hasContent "The paper describes a framework for KG embedding evaluation. It takes the generated vector representation and uses them for various tasks and report the corresponding results to see how these vectors perform in different tasks. Many different tasks are already implemented and a new embedding technique could be directly tested with this framework. The related work section is well written and also shows the differences to the proposed approach. Section three gives an overview of the framework and its extension points. Following I describe my procedure when installing and using the proposed software: I tried installing the software with pip but I received a FileNotFoundError because in the setup.py in line 10 it reads the file pip_readme.md which is not contained in the package but only in the GitHub repro. The authors should fix it, to allow others an easy installation. Moreover many users currently have anaconda installed. Maybe in the future a version on anaconda would be also helpful. With my updated version of the pip package I installed it in python 2 which works fine but in my new conda environment with python 3.8.1 I got a UnicodeDecodeError which I did not further analyzed. Since python 2 already reach its end of life [1], new software should target python 3 anyway. Afterwards I wanted to test main_00.py from the example folder. Unfortunately I couldn't find the file country_vectors.txt. Thus I proceed with main_01.py. There it couldn't load the FrameworkManager. The reason was, that in all examples the import statement is ```from evaluation_manager.manager import FrameworkManager``` but should be ```from evaluation_framework.manager import FrameworkManager``` After these changes, my script runs. The result directory is not generated at the correct place (as least when installing with pip) because in file evaluationManager.py line 228 [2], the results directory is the file path of evaluationManager.py. This results in a directory like "{{pythonpath}}\\lib\\site-packages\\evaluation_framework\\../results" where nobody would have a look. Thus I recommend to change to "os.getcwd()" for the current working directory. When looking at the results folder, only a log file is generated which describes that the data files are missing. Which is correct because in the pip file, no data files (like /evaluation_framework/Classification/data/Cities.tsv). Thus these files also needs to be included in the pip file. After manually copying these files, I first got some result files which are mostly empty. In the log I saw the error message: "Classification : Problems in merging vector with gold standard" The reason was that in the objectFrequencyS.txt file no URIs from the gold standard appeared. I thought that an example would cover such things. After trying out the software I asked myself where to get the KG for which I should generate the embedding. Based on the gold standard files I assume this is DBpedia. Maybe I have overlooked it but this should be clearly mentioned somewhere (and probably also in the GitHub readme). More importantly, not only the specific version of DBpedia is necessary but also which files [3] can be used to actually allow a comparison between the embedding techniques. I know that this does not be fixed by the evaluation framework as long as each embedding uses the same files, but maybe a general recommendation would be good. The software is not yet ready to be easily used but I think the authors can update it very fast. If this is done, the framework allows to easily compare different KG embeddings methods and I think that this solves an important gap. Some minor points: - Figure one can be converted to grayscale to allow a black and white print - It would also help to point out that this work is an extension? to [4] - page 4: "do not state it further"-> "do not state if further" ; "It takes in input a file" -> "It takes as input a file" - page 10: dbo:SportsTeam is over the line width because of \texttt [1] https://www.python.org/doc/sunset-python-2/ [2] https://github.com/mariaangelapellegrino/Evaluation-Framework/blob/master/evaluation_framework/evaluationManager.py#L228 [3] https://wiki.dbpedia.org/downloads-2016-10 [4] Pellegrino M.A., Cochez M., Garofalo M., Ristoski P. (2019) A Configurable Evaluation Framework for Node Embedding Techniques. In: Hitzler P. et al. (eds) The Semantic Web: ESWC 2019 Satellite Events. ESWC 2019. Lecture Notes in Computer Science, vol 11762. Springer, Cham After reading the rebuttal, I update my overall evaluation to accept. The technical details are solved."".
• Author.193.1 type RoleDuringEvent.
• Author.193.1 label "Marcin Pietrasik, 1st Author for Paper 193".
• Author.193.1 withRole PublishingRole.
• Author.193.1 isHeldBy Marcin_Pietrasik.
• b0_g523 first Author.193.2.
• b0_g523 rest nil.
• Author.193.2 type RoleDuringEvent.
• Author.193.2 label "Marek Reformat, 2nd Author for Paper 193".
• Author.193.2 withRole PublishingRole.
• Author.193.2 isHeldBy Marek_Reformat.
• Marcin_Pietrasik type Person.
• Marcin_Pietrasik name "Marcin Pietrasik".
• Marcin_Pietrasik label "Marcin Pietrasik".
• Marcin_Pietrasik holdsRole Author.193.1.
• Marek_Reformat type Person.
• Marek_Reformat name "Marek Reformat".
• Marek_Reformat label "Marek Reformat".
• Marek_Reformat holdsRole Author.193.2.
• Paper.193_Review.0_Reviewer type RoleDuringEvent.
• Paper.193_Review.0_Reviewer label "Anonymous Reviewer for Paper 193".
• Paper.193_Review.0_Reviewer withRole ReviewerRole.
• Paper.193_Review.0_Reviewer withRole AnonymousReviewerRole.
• Paper.193_Review.0 type ReviewVersion.
• Paper.193_Review.0 issued "2001-01-27T21:46:00.000Z".
• Paper.193_Review.0 creator Paper.193_Review.0_Reviewer.
• Paper.193_Review.0 hasRating ReviewRating.0.
• Paper.193_Review.0 hasReviewerConfidence ReviewerConfidence.4.
• Paper.193_Review.0 reviews Paper.193.
• Paper.193_Review.0 issuedAt easychair.org.
• Paper.193_Review.0 issuedFor Conference.
• Paper.193_Review.0 releasedBy Conference.
• Paper.193_Review.0 hasContent "This paper describes a method for taxonomy induction on knowledge graphs. The method is based on working with the knowledge graph as if it contained a collection of documents (subjects) which have annotations- tags- (property, object), so they can adapt and apply tag induction methods. The authors justify the need for extracting taxonomies from the knowledge graphs and summarize the state of the art by analyzing methods for taxonomy induction and tag induction. The paper contains a description of the method and its application to three datasets: Life, DBPedia and WordNet. The description of the experiment is not clear to me. It seems that in the three cases there are properties that define the taxonomy in the resource and that these properties are used as tags for the induction of the taxonomy, which could bias the results. The data used and the experiments have not been shared or their reproducibility facilitated. The results are compared with state of the art methods. The authors have used their own implementation of methods like Heymann and Garcia-Molina / Schmitz, which is a valuable effort but it is not clear if they are reproducing correctly the methods. This is also a limitation regarding the scalability analysis and the fact that some of these methods do not finish for some datasets. The text mentions the comparison with the tag induction methods, but Table 1 also includes the results of class taxonomy induction methods like Volker and Niepert, which is the one obtaining the best results. There is no discussion about these other methods, which are only applied to DBPedia, would they be applicable to the other ones? The results in Table 1 show that the method works better in some datasets but not in all, why?"".
• Paper.193_Review.1_Reviewer type RoleDuringEvent.
• Paper.193_Review.1_Reviewer label "Anonymous Reviewer for Paper 193".
• Paper.193_Review.1_Reviewer withRole ReviewerRole.
• Paper.193_Review.1_Reviewer withRole AnonymousReviewerRole.
• Paper.193_Review.1 type ReviewVersion.
• Paper.193_Review.1 issued "2001-02-09T17:53:00.000Z".
• Paper.193_Review.1 creator Paper.193_Review.1_Reviewer.
• Paper.193_Review.1 hasRating ReviewRating.2.
• Paper.193_Review.1 hasReviewerConfidence ReviewerConfidence.3.
• Paper.193_Review.1 reviews Paper.193.
• Paper.193_Review.1 issuedAt easychair.org.
• Paper.193_Review.1 issuedFor Conference.
• Paper.193_Review.1 releasedBy Conference.
• Paper.193_Review.1 hasContent "This work introduces a new method to induce a class taxonomy from knowledge graphs. The problem is interesting and relevant to the community. The main contribution is the idea of re-forming triples into document-tag tuples thus the commonly used word-frequency and co-occurrence techniques from NLP can be applied on knowledge graphs. Experimental results also demonstrate the merits of the proposed technique over traditional methods. Pros: (1). Comparing with popular machine learning techniques, this proposed work is straightforward and easy to implement, without sacrificing the performance. (2). The experiments are very well designed. For example, the experiment and following discussions on selecting the decay factor are not limited to the dataset used in the paper, but provide an insightful guidance to future users applying the approach. Also, experiments on dataset size would guide future user to make decisions using the approach as well. Cons: There are still some issues that could be improved in order to better demonstrate the work: (1). The methods seems to be a deterministic approach based on the description of the algorithm. Then why "we ran the methods five times on each dataset...."? (2). Any explanation on why 'Heymann and Garcia-Molina was not able to terminate sufficiently fast enough for us to obtain results on the Life dataset'? Also, it would be great if the author could discuss why the proposed method is faster to terminate than traditional methods. (3). I would suggest to also discuss about the validity of the assumption "subclasses will co-occur in document with their superclasses more often than with classes they are not logical descendants of", especially in the context of knowledge graph. (4). There are also many typos across the paper, the authors have to proofread it. Here are just some examples: (a). "high distribution across many topics" --> I guess you mean "high frequency" (b). "lower the more distant ..." --> "lower than the more distant..." (c). "It is preferred evaluation" --> 'it is a preferred evaluation' (d). "to derive the the harmonic mean between ..." --> duplicated "the""".
• Paper.193_Review.2_Reviewer type RoleDuringEvent.
• Paper.193_Review.2_Reviewer label "Anonymous Reviewer for Paper 193".
• Paper.193_Review.2_Reviewer withRole ReviewerRole.
• Paper.193_Review.2_Reviewer withRole AnonymousReviewerRole.
• Paper.193_Review.2 type ReviewVersion.
• Paper.193_Review.2 issued "2001-01-17T07:53:00.000Z".
• Paper.193_Review.2 creator Paper.193_Review.2_Reviewer.
• Paper.193_Review.2 hasRating ReviewRating.2.
• Paper.193_Review.2 hasReviewerConfidence ReviewerConfidence.4.
• Paper.193_Review.2 reviews Paper.193.
• Paper.193_Review.2 issuedAt easychair.org.
• Paper.193_Review.2 issuedFor Conference.
• Paper.193_Review.2 releasedBy Conference.
• Paper.193_Review.2 hasContent "The paper presents a method for inducing class hierarchies from knowledge graphs. The authors claim that the method is "simple" and is "scalable to large datasets". The method is based on ideas from tag hierarchy induction, i.e. counting classes and their co-occurrences. The approach is demonstrated based on different use cases with known datasets (Life, dbpedia, WikiData) and it is evaluated against other tag hierarchy induction methods. I like to "simpliness" of the approach allowing for performance and scalability while still showing convincing results. The paper is well written, the experiments have been clearly described and the data used for the experiments in publicly available."".
• Paper.194 type SubmissionsPaper.
• Paper.194 label "Claim Embeddings: an Empirical Study".
• Paper.194 title "Claim Embeddings: an Empirical Study".
• Paper.194 issued "2001-12-04T18:57:00.000Z".
• Paper.194 authorList b0_g524.
• Paper.194 submission Paper.194.
• Paper.194 track Track.Knowledge%20Graphs.
• b0_g524 first Author.194.1.
• b0_g524 rest b0_g525.
• Author.194.1 type RoleDuringEvent.
• Author.194.1 label "Valentina Beretta, 1st Author for Paper 194".
• Author.194.1 withRole PublishingRole.
• Author.194.1 isHeldBy Valentina_Beretta.
• b0_g525 first Author.194.2.
• b0_g525 rest b0_g526.
• Author.194.2 type RoleDuringEvent.
• Author.194.2 label "Andon Tchechmedjiev, 2nd Author for Paper 194".
• Author.194.2 withRole PublishingRole.
• Author.194.2 isHeldBy Andon_Tchechmedjiev.
• b0_g526 first Author.194.3.
• b0_g526 rest b0_g527.
• Author.194.3 type RoleDuringEvent.
• Author.194.3 label "Sébastien Harispe, 3rd Author for Paper 194".
• Author.194.3 withRole PublishingRole.
• Author.194.3 isHeldBy Sébastien_Harispe.

• first
• previous
• next