Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 866 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.1161. Min Act: -0.1700

Data Index: 5465847 (Open Web Text)

Max Activating Token Index: 120

Text #1

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0994. Min Act: -0.1700

Data Index: 7318788 (Open Web Text)

Max Activating Token Index: 87

Text #2

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.1052. Min Act: -0.1700

Data Index: 797327 (Open Web Text)

Max Activating Token Index: 81

Text #3

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0856. Min Act: -0.1700

Data Index: 210235 (Open Web Text)

Max Activating Token Index: 425

Text #4

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0817. Min Act: -0.1700

Data Index: 5233684 (Open Web Text)

Max Activating Token Index: 82

Text #5

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0639. Min Act: -0.1700

Data Index: 2534976 (Open Web Text)

Max Activating Token Index: 85

Text #6

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0648. Min Act: -0.1700

Data Index: 408252 (Open Web Text)

Max Activating Token Index: 69

Text #7

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0555. Min Act: -0.1700

Data Index: 5400900 (Open Web Text)

Max Activating Token Index: 54

Text #8

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0334. Min Act: -0.1700

Data Index: 1150659 (Open Web Text)

Max Activating Token Index: 54

Text #9

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0445. Min Act: -0.1700

Data Index: 3971833 (Open Web Text)

Max Activating Token Index: 97

Text #10

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0393. Min Act: -0.1700

Data Index: 2367701 (Open Web Text)

Max Activating Token Index: 108

Text #11

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0145. Min Act: -0.1700

Data Index: 7729969 (Open Web Text)

Max Activating Token Index: 78

Text #12

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0264. Min Act: -0.1700

Data Index: 8745007 (Open Web Text)

Max Activating Token Index: 74

Text #13

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0240. Min Act: -0.1700

Data Index: 4891453 (Open Web Text)

Max Activating Token Index: 81

Text #14

Max Range: 3.1161. Min Range: -3.1161

Max Act: 2.9943. Min Act: -0.1700

Data Index: 3742212 (Open Web Text)

Max Activating Token Index: 63

Text #15

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0076. Min Act: -0.1700

Data Index: 1163413 (Open Web Text)

Max Activating Token Index: 110

Text #16

Max Range: 3.1161. Min Range: -3.1161

Max Act: 3.0137. Min Act: -0.1700

Data Index: 6333439 (Open Web Text)

Max Activating Token Index: 136

Text #17

Max Range: 3.1161. Min Range: -3.1161

Max Act: 2.9901. Min Act: -0.1700

Data Index: 6228822 (Open Web Text)

Max Activating Token Index: 98

Text #18

Max Range: 3.1161. Min Range: -3.1161

Max Act: 2.9993. Min Act: -0.1700

Data Index: 2336873 (Open Web Text)

Max Activating Token Index: 77

Text #19

Max Range: 3.1161. Min Range: -3.1161

Max Act: 2.9983. Min Act: -0.1700

Data Index: 6786563 (Open Web Text)

Max Activating Token Index: 66