Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2916 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.6029. Min Act: -0.1700

Data Index: 1805692 (Open Web Text)

Max Activating Token Index: 733

Text #1

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.5585. Min Act: -0.1700

Data Index: 1487310 (Open Web Text)

Max Activating Token Index: 268

Text #2

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.5034. Min Act: -0.1700

Data Index: 3248272 (Open Web Text)

Max Activating Token Index: 224

Text #3

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.5218. Min Act: -0.1700

Data Index: 2896056 (Open Web Text)

Max Activating Token Index: 278

Text #4

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.5067. Min Act: -0.1700

Data Index: 7125013 (Open Web Text)

Max Activating Token Index: 553

Text #5

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4751. Min Act: -0.1700

Data Index: 4258469 (Open Web Text)

Max Activating Token Index: 971

Text #6

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4748. Min Act: -0.1700

Data Index: 1374217 (Open Web Text)

Max Activating Token Index: 343

Text #7

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4607. Min Act: -0.1700

Data Index: 3599636 (Open Web Text)

Max Activating Token Index: 412

Text #8

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4684. Min Act: -0.1700

Data Index: 2605328 (Open Web Text)

Max Activating Token Index: 465

Text #9

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4607. Min Act: -0.1700

Data Index: 7193803 (Open Web Text)

Max Activating Token Index: 650

Text #10

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4569. Min Act: -0.1700

Data Index: 604891 (Open Web Text)

Max Activating Token Index: 1018

Text #11

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4346. Min Act: -0.1700

Data Index: 624859 (Open Web Text)

Max Activating Token Index: 511

Text #12

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4526. Min Act: -0.1700

Data Index: 8169605 (Open Web Text)

Max Activating Token Index: 98

Text #13

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4239. Min Act: -0.1700

Data Index: 3610372 (Open Web Text)

Max Activating Token Index: 156

Text #14

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4224. Min Act: -0.1700

Data Index: 1355464 (Open Web Text)

Max Activating Token Index: 764

Text #15

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4106. Min Act: -0.1700

Data Index: 7028872 (Open Web Text)

Max Activating Token Index: 236

Text #16

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4527. Min Act: -0.1700

Data Index: 8423902 (Open Web Text)

Max Activating Token Index: 803

Text #17

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.3945. Min Act: -0.1700

Data Index: 4270127 (Open Web Text)

Max Activating Token Index: 589

Text #18

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4235. Min Act: -0.1700

Data Index: 934974 (Open Web Text)

Max Activating Token Index: 619

Text #19

Max Range: 4.6029. Min Range: -4.6029

Max Act: 4.4241. Min Act: -0.1700

Data Index: 3902757 (Open Web Text)

Max Activating Token Index: 386