Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2645 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.3038. Min Act: -0.1700

Data Index: 6187549 (Open Web Text)

Max Activating Token Index: 250

Text #1

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.2612. Min Act: -0.1700

Data Index: 5790313 (Open Web Text)

Max Activating Token Index: 760

Text #2

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.2104. Min Act: -0.1700

Data Index: 3853086 (Open Web Text)

Max Activating Token Index: 971

Text #3

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.2054. Min Act: -0.1700

Data Index: 1022332 (Open Web Text)

Max Activating Token Index: 391

Text #4

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1834. Min Act: -0.1700

Data Index: 872010 (Open Web Text)

Max Activating Token Index: 220

Text #5

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1293. Min Act: -0.1700

Data Index: 80405 (Open Web Text)

Max Activating Token Index: 173

Text #6

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1512. Min Act: -0.1700

Data Index: 586927 (Open Web Text)

Max Activating Token Index: 321

Text #7

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1251. Min Act: -0.1700

Data Index: 4472596 (Open Web Text)

Max Activating Token Index: 800

Text #8

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1773. Min Act: -0.1700

Data Index: 8111066 (Open Web Text)

Max Activating Token Index: 545

Text #9

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1153. Min Act: -0.1700

Data Index: 5423548 (Open Web Text)

Max Activating Token Index: 749

Text #10

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1111. Min Act: -0.1700

Data Index: 6257789 (Open Web Text)

Max Activating Token Index: 736

Text #11

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0947. Min Act: -0.1700

Data Index: 1231280 (Open Web Text)

Max Activating Token Index: 891

Text #12

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0895. Min Act: -0.1700

Data Index: 1202936 (Open Web Text)

Max Activating Token Index: 239

Text #13

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0787. Min Act: -0.1700

Data Index: 424742 (Open Web Text)

Max Activating Token Index: 706

Text #14

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0925. Min Act: -0.1700

Data Index: 6625521 (Open Web Text)

Max Activating Token Index: 93

Text #15

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0876. Min Act: -0.1700

Data Index: 8643033 (Open Web Text)

Max Activating Token Index: 883

Text #16

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0846. Min Act: -0.1700

Data Index: 2347907 (Open Web Text)

Max Activating Token Index: 578

Text #17

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.1104. Min Act: -0.1700

Data Index: 8094579 (Open Web Text)

Max Activating Token Index: 809

Text #18

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0676. Min Act: -0.1700

Data Index: 7073205 (Open Web Text)

Max Activating Token Index: 535

Text #19

Max Range: 4.3038. Min Range: -4.3038

Max Act: 4.0409. Min Act: -0.1700

Data Index: 4801946 (Open Web Text)

Max Activating Token Index: 259