Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1494 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.7617. Min Act: -0.1700

Data Index: 655858 (Open Web Text)

Max Activating Token Index: 505

Text #1

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.6542. Min Act: -0.1700

Data Index: 890161 (Open Web Text)

Max Activating Token Index: 41

Text #2

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.6588. Min Act: -0.1700

Data Index: 3593881 (Open Web Text)

Max Activating Token Index: 669

Text #3

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.6205. Min Act: -0.1700

Data Index: 5013430 (Open Web Text)

Max Activating Token Index: 845

Text #4

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.6101. Min Act: -0.1700

Data Index: 5213088 (Open Web Text)

Max Activating Token Index: 86

Text #5

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5887. Min Act: -0.1700

Data Index: 8659010 (Open Web Text)

Max Activating Token Index: 39

Text #6

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5952. Min Act: -0.1700

Data Index: 8690289 (Open Web Text)

Max Activating Token Index: 551

Text #7

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5854. Min Act: -0.1700

Data Index: 8795835 (Open Web Text)

Max Activating Token Index: 754

Text #8

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5772. Min Act: -0.1700

Data Index: 2277584 (Open Web Text)

Max Activating Token Index: 365

Text #9

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5739. Min Act: -0.1700

Data Index: 631811 (Open Web Text)

Max Activating Token Index: 244

Text #10

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5651. Min Act: -0.1700

Data Index: 2721376 (Open Web Text)

Max Activating Token Index: 563

Text #11

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5623. Min Act: -0.1700

Data Index: 1662182 (Open Web Text)

Max Activating Token Index: 479

Text #12

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5463. Min Act: -0.1700

Data Index: 5831897 (Open Web Text)

Max Activating Token Index: 936

Text #13

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5576. Min Act: -0.1700

Data Index: 3560484 (Open Web Text)

Max Activating Token Index: 871

Text #14

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5543. Min Act: -0.1700

Data Index: 7038196 (Open Web Text)

Max Activating Token Index: 436

Text #15

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5380. Min Act: -0.1700

Data Index: 4782976 (Open Web Text)

Max Activating Token Index: 696

Text #16

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5865. Min Act: -0.1700

Data Index: 8273021 (Open Web Text)

Max Activating Token Index: 363

Text #17

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.4918. Min Act: -0.1700

Data Index: 791179 (Open Web Text)

Max Activating Token Index: 185

Text #18

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.4978. Min Act: -0.1700

Data Index: 4582574 (Open Web Text)

Max Activating Token Index: 371

Text #19

Max Range: 3.7617. Min Range: -3.7617

Max Act: 3.5163. Min Act: -0.1700

Data Index: 6347081 (Open Web Text)

Max Activating Token Index: 838