Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1957 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1955. Min Act: -0.1700

Data Index: 8008672 (Open Web Text)

Max Activating Token Index: 917

Text #1

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.2298. Min Act: -0.1700

Data Index: 6037505 (Open Web Text)

Max Activating Token Index: 787

Text #2

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1578. Min Act: -0.1700

Data Index: 8481467 (Open Web Text)

Max Activating Token Index: 625

Text #3

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1506. Min Act: -0.1700

Data Index: 5860668 (Open Web Text)

Max Activating Token Index: 389

Text #4

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1216. Min Act: -0.1700

Data Index: 5134294 (Open Web Text)

Max Activating Token Index: 481

Text #5

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1092. Min Act: -0.1700

Data Index: 6180479 (Open Web Text)

Max Activating Token Index: 921

Text #6

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1958. Min Act: -0.1700

Data Index: 4814371 (Open Web Text)

Max Activating Token Index: 811

Text #7

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.1016. Min Act: -0.1700

Data Index: 4846904 (Open Web Text)

Max Activating Token Index: 625

Text #8

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0827. Min Act: -0.1700

Data Index: 1806990 (Open Web Text)

Max Activating Token Index: 524

Text #9

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0967. Min Act: -0.1700

Data Index: 5901586 (Open Web Text)

Max Activating Token Index: 316

Text #10

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0753. Min Act: -0.1700

Data Index: 4458249 (Open Web Text)

Max Activating Token Index: 81

Text #11

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0759. Min Act: -0.1700

Data Index: 963287 (Open Web Text)

Max Activating Token Index: 651

Text #12

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0959. Min Act: -0.1700

Data Index: 1966543 (Open Web Text)

Max Activating Token Index: 128

Text #13

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0768. Min Act: -0.1700

Data Index: 1627006 (Open Web Text)

Max Activating Token Index: 740

Text #14

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0807. Min Act: -0.1700

Data Index: 8726507 (Open Web Text)

Max Activating Token Index: 261

Text #15

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0491. Min Act: -0.1700

Data Index: 6564973 (Open Web Text)

Max Activating Token Index: 881

Text #16

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0468. Min Act: -0.1700

Data Index: 4469758 (Open Web Text)

Max Activating Token Index: 55

Text #17

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0473. Min Act: -0.1700

Data Index: 1144238 (Open Web Text)

Max Activating Token Index: 135

Text #18

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0520. Min Act: -0.1700

Data Index: 131361 (Open Web Text)

Max Activating Token Index: 979

Text #19

Max Range: 3.2298. Min Range: -3.2298

Max Act: 3.0377. Min Act: -0.1700

Data Index: 4405785 (Open Web Text)

Max Activating Token Index: 440