Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1096 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.6248. Min Act: -0.0000

Data Index: 1432276 (The Pile)

Max Activating Token Index: 665

Text #1

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.6796. Min Act: -0.0000

Data Index: 25556 (The Pile)

Max Activating Token Index: 538

Text #2

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.6688. Min Act: -0.0000

Data Index: 1280133 (The Pile)

Max Activating Token Index: 1015

Text #3

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.4702. Min Act: -0.0000

Data Index: 1607725 (The Pile)

Max Activating Token Index: 846

Text #4

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.5398. Min Act: -0.0000

Data Index: 1840472 (The Pile)

Max Activating Token Index: 769

Text #5

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.3495. Min Act: -0.0000

Data Index: 735881 (The Pile)

Max Activating Token Index: 793

Text #6

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.1979. Min Act: -0.0000

Data Index: 1151011 (The Pile)

Max Activating Token Index: 76

Text #7

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.4051. Min Act: -0.0000

Data Index: 1457705 (The Pile)

Max Activating Token Index: 30

Text #8

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.2810. Min Act: -0.0000

Data Index: 962793 (The Pile)

Max Activating Token Index: 751

Text #9

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.3056. Min Act: -0.0000

Data Index: 500562 (The Pile)

Max Activating Token Index: 1023

Text #10

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.2426. Min Act: -0.0000

Data Index: 24279 (The Pile)

Max Activating Token Index: 631

Text #11

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.1792. Min Act: -0.0000

Data Index: 556320 (The Pile)

Max Activating Token Index: 546

Text #12

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.3532. Min Act: -0.0000

Data Index: 663726 (The Pile)

Max Activating Token Index: 280

Text #13

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.3694. Min Act: -0.0000

Data Index: 4487 (The Pile)

Max Activating Token Index: 841

Text #14

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.2785. Min Act: -0.0000

Data Index: 834787 (The Pile)

Max Activating Token Index: 516

Text #15

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.3302. Min Act: -0.0000

Data Index: 1056437 (The Pile)

Max Activating Token Index: 890

Text #16

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.2074. Min Act: -0.0000

Data Index: 353181 (The Pile)

Max Activating Token Index: 770

Text #17

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.1253. Min Act: -0.0000

Data Index: 1526802 (The Pile)

Max Activating Token Index: 1017

Text #18

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.3208. Min Act: -0.0000

Data Index: 1117586 (The Pile)

Max Activating Token Index: 522

Text #19

Max Range: 1.6796. Min Range: -1.6796

Max Act: 1.2987. Min Act: -0.0000

Data Index: 1689683 (The Pile)

Max Activating Token Index: 995